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During the early years of the Society, its greatest strength lay in the Sections 
and Regional organizations, many of which antedated the formation of the national 
organization. The first four national conventions were held in conjunction with 
regional conferences (197-1950). 


A major step forward was taken in 1951 when, for the first time, the Annual 
Convention was held as an independent activity of the Society, completely divorced from 
any connection with regional conference. Under the leadership of Wade R. Weaver, 
the American Society for Quality Control began to emerge as an organization capable of 
serving the national interests of its members in ways beyond the limitations of section 
and regional groups. A General Convention Committwe was created and given the respon- 
sibility to provide the "ong range guidance and planning needed to insure the pro- 
gramming and management of effective annual conventions. This Committee served the 
Society ably, as the record of nine consecutive conventions under its surveillance 
amply testify. 


With completion of the 1959 Convention, the General Convention Committee was dis- 
continued, thus completing a transition program begun a year earlier to pave the way 
for the next major step forward in convention management for ASQC, viz, the transfer 
of the business management of its conventions to the Society headquarters office and 
the program planning to a committee under the supervision of a vice-president of the 
Society. The 1960 Convention was the first one to be managed and programmed under 
this new administrative procedure. 


These Transactions constitute a record of the presentations at the Fifteenth 


Annual Convention. They have not been subjected to review by the Editorial Board 

of the Society. The timetable for planning our conventions precludes this at the pre- 
sent time. After the Convention, the papers will be reviewed by the Editorial Board 
and certain of them may appear in Industrial Quality Control. 


, Request for permission to reprint any portion of these Transactions should be 
addressed to Professor Irving W. Burr, Chairman, Editorial Board, Statistical Lab- 
oratory, Purdue University, Lafayette, Indiana. 


Although these Transactions are copyrighted, the American Society for Quality 
Control assumes no responsibility for any of the authors' statements. Responsibility 
for the content of each paper resides with its author. 


Mason E. Wescott 
Retiring Chairman, Editorial Board 
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HOW TO EVALUATE A COMPANY'S QUALITY CONTROL NEED 


Leonard A. Seder 
Leonard A. Seder and Associates 


The object of this paper is to attempt to answer the question: "How does one evalu- 
ate a company's quality control need?" Considering the wide varieties of products, 
sizes of companies and specific technological problems, one might well find it difficult 
to generalize. There is no question, for example, that different companies have widely 
different types of quality problems, because of the varying nature of the products they 
make. Moreover, F quality control needs vary widely, also, if we consider the word 
"need" in its narrow sense of specific need for particular measuring techniques, statis- 
tical methods, or process control devices, However, if we consider the word "need" in 
its broader sense of functions, attitudes and philosophies then it does turn out that 
most companies’ quality needs can be stated in similar terms. 


company Quality 
s is so ause, the broad sense, most companies' quality objectives are sin- 
ilar. A. V. Feigenbaum's statement of these objectives in his discussion of Total 


Quality Control (1) is quite apt. He says the objectives of a quality function should 
be: 


1. To provide assurance that the company's products are right for 
the customer. 
2. To assist in achieving Optimum Quality Costs for these products. 


Assuming that these are, in fact, the company's objectives for its quality func- 
tion, we shall discuss, in terms of these objectives, how to evaluate the quality con- 
trol need of a specific company through the gathering and interpretation of certain key 
data, the interviewing of key individuals in the organization and the personal observa- 
tion of the company's quality control functions in operation. 


The intent will be to make the evaluation as objective as possible. To this ex- 
tent, certain yardsticks or rough rules of thumb will be suggested. Slavish adherence 
to such yardsticks, however, is obviously dangerous. These figures are not to be taken 
as "official" or the "national average". In spite of the effort at objectivity, it is 
obvious that some factors must--at the present state of knowledge--remain subjective. 
In these areas, there is evidently room for differences of opinion, even among the "ex- 
perts". The discussion of such subjective factors herein must be considered as the 
opinions of the author. 


We shall consider, then, such questions as: 

A. How does one go about evaluating a company's quality control need? 

B. What measurements can be made on the overall status that yield an opportunity 
to stack up the company's status against others? 

C. What observations should be made to read the “pulse” of the organisation? 

D. What symptoms indicate the possible presence of underlying unsolved problems? 


In effect, it is as though I were an elderly physician addressing other physicians 
some 50 or 60 years ago, before the development of modern diagnostic sests. I am going 
to try to summarize my "vast" experience and tell you to take the patient's pulse, blood 
pressure and temperature, look down his throat, and up his nose, hammer on his knees, 
peer at his eyeballs, and how to come up with a diagnosis of appendicitis, liver ail- 
ment, rupture or what-not. I do not really jest. Some years from now--less than 50, 
I hope--we shall have much better tests and measurements for evaluating these quality 
control needs. Today, our means are crude, but they are nevertheless worth discussing. 


Much of what follows was learned through fortunate association with J. M. Juran, 
who has done the pioneering work and continues to lead our thought in these management 
areas. 


Assurance Of Customer Satisfaction 

eigenbaum's first objective was "to provide assurance that the company's quality 
is right for the customer." Note that this broad statement includes assurance of 
"quality of design" as well as "quality of conformance". How, then, can we evaluate 
the company's: achievement (or lack of it) with respect to this objective? Clearly, we 
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need a measure of customer satisfaction. More properly, the pa a needs such a meas- 
ure, which is presented monthly to top management and used by guide decisions in 
the customer quality relations area. The absence of such a measure is, in itself, a 
symptom, for its absence generally is accompanied by a disillusioned Sales Department, 
and by management stampedes on the basis of individual complaints. From the manufac- 
turing view, these are regarded as over-exaggeration of “isolated cases". If no such 
measure exists, the first step in evaluation is to obtain it in one of several ways: 
1. Determining the rate of rejection, return or complaint by customers. 
2. Determining the field reliability of its product against desired goals 
and against competition. 
3. Determining the actual outgoing quality by a customer-oriented sampling 
and performance testing against government, industry or self-imposed 
standards. 


In addition to such direct measures, some indirect indicators of customer satis- 
faction are obtainable from: 
1. The results of customers' “vendor survey team" reports. 
2. The opinions of the company Sales Department people. 
3. —, actual customers, or ultimate consumers (when the two are 
different. 


Perhaps the most universal of these measures is that of the rate of customer re- 
turn or complaint. A few words are therefore in order about the preparation and inter- 
pretation of such data. 


First, it must be recognized that returns an! complaints are as valid measure of 
customer satisfaction only under particular conditions. It is well known that the 
boiling point of the customer (i.e. the point at which he will return or complain in 
writing about an unsatisfactory product) varies directly with the ce of the product 
and the economic climate. Juran has pointed out that there is a curve relating the 
"ratio of complaints to product failures" to the "unit price" of the product and that 
this curve shifts to the right in a seller's market, to the left in a buyer's market. 
In other words, the number of complaints received by a razor blade manufacturer will 
normally be 1/1000th or less of the mmber of "failures" in the product shipped, while 
the automobile manufacturer may receive 50% or so, and the missile-maker more nearly 
100%. All these numbers are smaller when the goods are hard to get. 


This fact means that number of complaints are a valid measure of customer satis- 
faction only if the company's products are at the high end of the curve, whereas they 
are poor measure--because they are too insensitive to product quality changes~--if the 
product is at the low end of the scale. In such case, some other means must be found, 
such as outgoing product examination, to measure failures directly and thus to estimate 
probable customer satisfaction. 


The complaint rates of various companies in different industries confirm this fact. 
The following is a tabulation from the author's experience. 


TYPICAL COMPLAINT OR RETURN RATES 
Consumer Products 


Time to Consume 
or Wear Out ical Products Typical Complaint Rate 
Years Appliances, autos, furniture, 

clocks, watches, cameras 1 -10 £ 

Months Shoes, garments, cigarette 
lighters, medical products eal -1 

Days Razor blades, candy, food, 
cosmetics, drugs .05 
(10 - 500 per million) 

Industrial Products 
Unsophisticated customers 5 <5 
Sophisticated customers 1 -10 & 


In preparing figures to measure a company's complaint rate, it is first important 
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to make sure that: 
1. Sales Policy returns are excluded. 
2. There is an unbiased classification of whether the customer or the prod- 
uct is at fault. 


Beyond obtaining and evaluating such a specific measure as complaint rate, there 
are a number of other "subjective" factors which ought to be considered in order to ar- 
rive at a judgment as to whether the company is sensitive to customer needs and is alert 
to the advantages that can be had from plugging in to the stream of information that 
comes back from the users of the product. The following is a symptom checklist to 
evaluate such alertness: 

1. Is there a procedure for handling the customer who complains so that his 
dissatisfaction can be minimised? nie 

2. Is there a formal procedure for investigating the cause of complaint, re- 
quiring an answer in writing that is sent to the customer or at least 
filed? 

3. Are the accumulated complaint data summarised to detect chronic situa- 
tions, to determine the effect of deliberate (or even accidental) changes 
in design or manufacturing methods, to learn of any peculiar geographic 
anomalies in the complaint rate, etc.? 

4. Is all complaint information, including both individual complaints and 
analyses thereof, fed back to the internal company function that can take 
steps to remedy it? 

S. Is there an alertness to the significance of an unusual complaint, even 
though it may seem to be an "isolated case" or otherwise not be statis- 
tically significant? Such unusual complaints that "can't happen" may be 
highly important to the company's future quality reputation. 

6. Is there an awareness generally of the principal categories of complaint? 
Do they match up with the principal causes of factory rejection? If they 
do not, it may mean that unnecessary cost is being incurred to find de- 
fects the customer is not complaining about, while at the same time in- 
sufficient attention is being paid to the defects that really bother him. 

7- In the case of industrial t s, have the standards for non-measur- 
able defects been discussed and agreed to with the customer's quality con- 
trol people? Is there a method of transmitting this agreement to all 
the people who need to know it? 

8. In the case of consumer products, have efforts been made to learn the 
real sensitivities of the customer, or are we instead depending on the 
opinions of the factory "experts" who are highly sensitized to defects in 
the product that the customer may not be able to see, or smell, or taste? 
If so, we may be unrealistically tight in attempting to achieve perfec- 
tionism without accompanying value. 

9. At the other end of the scale, is the customer actually unreasonable in 
his quality demands? It may actually be better not to do business with 
a customer who insists on standards or AQLs which are clearly well beyond 
our ability to meet. 


Assurance Of Optimum Quality Cost. 

Feigenbaum's sec objective was: "To assist in achieving optimum quality costs 
for these products." Our survey of the company's quality control need must, then, 
concern itself with how well the optimum has been achieved or is being sought. 


The first question is whether the company has a knowledge of its Quality Costs. 
The absence of such knowledge in reasonable detail is, in itself, a serious symptom. 
For it is obviously essential to have a continuing knowledge of Quality Costs if we are 
to talk about striving for an optimum Quality Cost. Quality Costs are all of the 
costs and losses which exist b it is ry to meet customers' requirements or 
quality specifications. They are conveniently divided into: 
Failures - The losses in scrap, rework, repair, discounts, returns 
associated with material which failed to meet requirements. 
Appraisal - The costs of sampling, measuring, testing, segregating, 
sorting to appraise quality and to keep product streans 
Prevention- The costs of planning, analysis, diagnosis, study and 
remedy-generation to improve existing or future quality 
levels. 


The meaning of "optimum" Quality Costs requires an understanding of the economics 
of quality. Failure costs are high when the conformance of product to requirements is 
poor; they decrease as better conformance is achieved. Prevention costs, on the other 
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hand, rise as better and better conformance is sought. Appraisal costs have aspects 
of both behaviors. Those Appraisal costs which are mainly sorting and segregating are 
reduced with better conformance, while those Appraisal costs which are preventive in 
nature tend to increase with better conformance. 


We have, in effect, one curve which descends with improving quality, another which 
ascends, and a third U-shaped curve. The optimum occurs when the sum of the three 
curves is at a minimum. In other words, optimum Quality Cost is achieved when the sum 
of Failure, Appraisal and Prevention costs is at a minimum. This is a lot easier to 
describe in theoretical terms than to determine in practice. However, some approaches 
which have been tried and some approximate indexes may be helpful. 


The problem is, how do we know on which side of the minimum we are or whether we 
are right at the minimum? For that question, we have some approximate criteria: 

1. Juran: When Quality Costs are in the range of $500 - $1,090 per 
productive employee per year, or higher, it is likely that the 
company stands to gain by spending more on Prevention. 

26 pat nd Quality Costs exceeding 10% of Sales are usually 

@. Moreover, if Prevention costs are less than 10% 

of the total Quality Costs, we can expect to reduce Failures 
and Appraisal by spending more on Prevention. 


The decision to try to reduce Quality Costs by increasing the amount spent on Pre- 
vention should not be made by comparing the company's performance with these indexes. 
The absolute tude of the Failure and Appraisal Costs will, in themselves, justify 
or not justi ie cost of a program to reduce them. It is useful, however, to know 
that others have been successful in making such reductions at the levels indicated in 
the foregoing figures. 


How Prevention Honey Should Be es 
se exes do no ie whole story, by any means, for the money being spent 


on Prevention may not be spent in the best way. In evaluating the company's quality 
control need, therefore, we must go into further detail on how the Prevention money is 
spent. 


Here is a checklist of symptoms to look for that indicate whether an apparently 
uate Prevention budget is being used in the right way. It will also be a useful 
the Prevention budget is inadequate in order to check for missing items. 

1. Is there a clean separation between those people who do the day-to-day 
decision-making and those people who do the long-range analytical and 
planning work necessary to Prevention? 

2. Does the organisational structure recognize the essential principles of 
organizaing for a breakthrough? In a word, is the team approach em- 
ployed in planning and following through the program? (2) 

3. Do the people assigned to Prevention recognize the five basic principles 
necessary to accomplishment of Prevention goals? (3) 

a. Principle of Prevention 
be Principle of Scientific Method 
Ce Principle of Separating Vital from Trivial 
d. Principle of Staff Assistance 
on Principle of Coordination 

me: lk. Are the chronic defects tackled, rather than the sporadic? 

» 5. Is the distinction made between controllable and operator- 
controllable defects and is the appropriate program of improvement 
employed for each type? 

6. Are the basic dats, such as process capabilities, determined and made 
available to the engineering and planning functions? 

cae 7e Is adequate quality planning done on new products so that expensive 

fiascos are avoided? 
8. For complex products, is there an independent evaluation of design 
before release to Production? 
9. Are adequate steps taken to plan for reliability tests and data 
: feedback? 


Are Appraisal Costs Too ? 

e have cated, » how the question of achievement of optimum Quality Cost 
can be evaluated in several ways--some of them objective and some subjective. The 
discussion thus far is based on a premise; namely, that the dominating element of Qual- 
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ity Cost is the Failure cost. This is the usual case, since Failure cost generally 
amounts to 70% or more of the total Quality Cost. Occasionally, however, one encoun- 
ters a situation where the Appraisal cost is the dominating element of Quality Costs, 
amounting to more than half of the total. The question may then properly be raised as 
to whether we are beyond the optimm, or, in other words, are over-inspecting. 


Here, indexes of the type often cited may do more harm than good, i.e. the type of 
index which is expressed in terms of: 
Dollars of Inspection Cost i Number of Inspectors 
Dollars of Mfg. Cost Number of Operators 


The fallacy here is the assumption that the Inspection cost should vary with the 
amount of productive effort, whereas, in fact, it should be primarily dependent on the 
frequency of failures, or the potential frequency of failures. 


Instead of depending on such indexes, we should study the situation surrounding 
each of the major Appraisal costs, since the answer to our question will depend on the 
facts of the individual situations. We can divide the question into two basic situa- 
tions: 

1. When the Appraisal costs are mainly in sorting and segregating good 
from bad 


2. When the Appraisal costs are mainly in sampling, either of the process 
or the product. 


The first of these is fairly easy to judge. When the major Appraisal costs are 
in detecting and removing defects, the cost of not removing them should be determined 
and compared with the exisiing cost of removing them. The cost of not removing them 
will usually be found to consist of repairs, wasted operations, teardown and rebuild, 
production-line downtime, field failures, customer returns and so on. The costs of 
such difficulties are usually calculable, and the inspection data gives a measure of 
the frequency with which such a difficulty will be encountered if the sorting operation 
is replaced with sampling or abolished altogether. The decision as to the need can 
then be made on a rational economic basis. It is important here not merely to deter- 
mine the total frequency of such failures, but also the pattern in which they occur. 

A highly erratic pattern from lot-to-lot means that a sampling plan can be depended on 
to detect the bulk of them. 


It is not quite so easy to judge whether Appraisal costs are excessive when they 
are mainly in sampling of the process or product. This is because the inspection may 
well be acting so as to prevent the incidence of defects which would otherwise result in 
Failure costs far exceeding the existing Appraisal costs. Thus, it is argued, we can't 
answer the question without experimenting in reduction of inspection, which may be at 
great danger. It is like debating the question of whether reducing the local police 
force by 25% would actually result in a significant increase in the crime rate. Re- 
flection will indicate that this depends a good deal on the morality of the local cit- 
isenry. Under stress, some companies have tried the brute force method of arbitrarily 
eliminating or reducing such sampling inspection only to have the whole thing boomerang 
in time. The insidious thing is that such a move may appear to work when viewed in the 
short range, because the full effect is not apparent until the good habits engendered by 
the "policing" influences are slowly replaced with the bad ones of unbridled license. 


A far better approach is to examine the facts surrounding the specific situations 
to see whether factors have been overlooked which will permit discriminating reduction 
in inspection. Here is a checklist of possibilities: 

1. Is anything being done to prevent the recurrence of rejected lots? 

If not, can it be? This should inevitably reduce inspection costs. 

2. Are rejected lots actually being accepted anyway, via a Material Review 
or similar procedure? If so, action to change the specifications is 
indicated and inspection costs may be reduced. 

3. Is there a combination of the last two, viz. ultimate acceptance of 
rejected material and no action to prevent recurrence? The inspec- 
tion cost in such instances is clearly a waste of money. 

le. Is advantage being taken of good history? Is reduced or skip-lot 
inspection being used when failures are relatively rare? 

5. Can the inspection be done earlier before the streams have become 
contaminated? 

6. Can the inspection be done later without adding much to the Failure 
Costs? 

7. If process sampling is being done and decisions made by inspectors, 
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could this not be better done by operators, at less cost and with 
greater effect? 

8. Can improvements be made in the basic efficiency of the inspection 
operation? Work simplification techniques are applicable to in- 
spection as well as production operations. Possibly automatic in- 
spection is the more economic answer to a chronic defect situation 
that has no hope of improvement. 


In summary, the minimum amount of inspection is that amount necessary to verify 
that quality is and continues to be satisfactory. Whether we are concerned with in- 
coming, in-process or outgoing product, such verification should be able to be accom- 
plished with an audit type of inspection rather than a lot-by-lot acceptance and re- 
jection. If more inspection than that is being done, it is either because the quality 
does not continue to be satisfactory over a period of time or because Quality management 
is not responsive to the changing conditions. The conclusions are thereby obvious. 


Other Evaluation Points 
¢c points for evaluation include certain specific procedures and practices 
which historically have been found to contribute to the meeting of the basic quality ob- 
jectives. If one or another of these is missing, it does not necessarily mean that 
there is a need, but its absence should at least raise the question. In each of the 
following cases, the question is whether there exists an adequate plan, and, if so, 
whether it is being carried out effectively. 
1. A plan for control of the accuracy of measuring instruments and gags. 
2. A plan for handling of deviations such that decisions are made at 
higher levels and information is utilized for improvement. 
A plan for learning the level of competitive quality. 
4. A plan for conveying to production operators accurate information on: 
which are the vital quality characteristics, what are the tolerances 
on process variables and which process specifications are mandatory 
and which optional. 
5. A plan for conveying to vendors the results of quality measurements 
obtained on their shipments. 
6. A plan for providing management with an executive instrument panel 
on quality. 


All of these, and others I have omitted, add up to the need for a written manual 
describing the company's practices in detail. The manual is not only the proof that 
the answers have been thought out, but it is also the way of progressing from "govern~ 
ment by men" to "government by law." 


Evaluation Of Intangible Factors 

Thus far, we have considered a company's quality control need as expressed in 
terms of its basic quality objectives, as defined herein. Insofar as possible, we 
have tried to consider either the basic economics or some other fairly tangible factors. 
There are, however, a number of other factors which have an effect on the achievement of 
the company quality objectives and yet are not measurable or tangible. There are, for 
example, the questions of organization for quality, the managerial and statistical com- 
petence of the key individuals in the organisation, and the general state of quality 
morale. That these factors have an important effect on the achievement of quality ob- 
jectives is undeniable, yet there is also room for considerable difference of opinion in 
evaluating the company's adequacy in these areas. 


Despite such differences in opinion, there is need to set down some fundamental 
needs in these areas. By implication, the company which has not faced up to these 
needs should be urged to do so. 


With regard to organization for quality, experience has shown that there are many 
different forms of organization which can successfully accomplish the objectives. It 
is not so much the form of organization that needs critical evaluation as it is the 
question of whether the company has a sound plan of satisfying its basic organizing 
needs. These basic needs are set forth below. 

1. Organisation for Product arene es - With some few exceptions, there is 
usua. a ne or wo approval of product for shipment to cus- 
tomers to be done by a group which is independent of the people responsi- 
ble for making the product. The absence of such an independent final 
— eee found to be the underlying cause of the symp- 
toms of: 

a. A higher-than-necessary level of customer dissatisfaction because of 
the tendency of the "maker" to solve day-to-day delivery and cost 
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problems at the expense of quality. 

be. Lack of adequate data for feedback to prevent recurrence, to identify 
major quality problems, to furnish executive reports on quality. 

ce A morale problem as the result of frustration on the part of those 
seriously attempting to control process quality. 

ea for Process Control - There is a need for the clear delega- 

on of aw y for mg quality decisions during manufacture to the 

end that everyone involved knows and understands his responsibilities. 

To be at all effective, this must usually be a written delegation, em- 

bodying clear statements as to: 

a. Who decides whether a xr chine or process shall commence, or continue, 
to run? His is the responsibility for defects made. 

b. Who decides that lots or batches emerging from an operation may con- 
tinue or to have the next operation performed? His is the respon- 
sibility for defects missed. 


Organization for Solving - e Problems - There is a need for one or 
more 8, depending on sise, from the day-to-day ac- 


tivities to devote their efforts to the identification of the major 

chronic quality problems, to the development of programs to solve these 

problems, and to the planning of quality on new products so as to avoid 

these problems in the future. The absence of a separate person or 

group so charged means that it is unlikely that these jobs are being 

done, so that the company is, at best, just “holding its own" without 

making any significant progress in the quality area. 

Managerial and Statistical © tence - There is a need for a Quality 

who has the perspective of Total Quality Control, as op- 

posed to great skill in any one of the particular techniques for achiev- 

ing it, and who has demonstrable ability as a manager. The extent of 

statistical skills needed in the organization depends to a large extent on 

the size and type of quality problems encountered, but seldom is the total 

absence of these skills tolerable in today's competition. 

Quality Morale - There is always the need for a high level of quality 

dorale- throughout the organization. No company can long survive the 

widespread conviction among its employees that "management" or some power- 

ful group within management "does not really care about quality." The 

causes for such conviction are many and sometimes fairly complex. Among 

the more common are: 

a. Lack of a clearcut management policy on quality. 

be. Habitual exceeding of product specifications without penalty or alarm. 

Cc. Obvious fluctuation of quality standards with changing inventories or 
production schedules. 

d. Daily exposure to management decisions which are unashamedly directed 
to taking chances on the shipping of marginal quality. 

e. Constant pressure on operators for improvement when it is obvious to 
them that the major improvement can be made only by some action on the 
part of management. 


The Essence Of Evaluation 
nm Summary, we have looked at a number of checkpoints for evaluating a company's 
quality control need. The main headings were: 


How well is the customer being satisfied? 

Is enough Prevention Money being spent to avoid uneconomic 
Failure costs? 

Is real Prevention being done through proper organizing and 
competency of people? 

Is amount of inspecticn appropriate? 

Is the organization for quality well thought out and does it 
satisfy basic needs? 

Is there good level of quality Morale? 


IT. Yeigenbaum, A. V., "The Challenge of Total quality Control", Industrial Quality 
Control, vol. XIII, No. 11, May, 1957. 
2.  YJuran, J. Me, "Industrial Diagnostics", The Management Review, American Management 
Association, June, 1957. 
3. Juran, J. Me, Quality-Control Handbook, Page 22, McGraw-Hill, 1951. 
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QUALITY COST CONTROL 


Warren R. Purcell, Director of Quality Control 
Raytheon Company, Lexington, Massachusetts 


From modest beginnings as a new field of activity on the industrial scene, 
Qua 1"; Control has developed into a major engineering profession. It started with 
the technique of the control chart; it now encompasses the planning and administra- 
tion of the entire quality control system, starting with the design review and continu- 
ing through incoming material control, process control, and finished product accep- 
tance to the processing and analysis of data on field results and the monitoring of 
corrective action. It started with the application of scientific sampling to inspection; 
it now covers the design and construction of inspection and test equipment, its 
calibration and maintenance, the planning and implementation of its use, and break- 
throughs to new frontiers in the field of measurement. It started by reporting "out- 
of-control" conditions; it now provides special skills needed to diagnose the quality 
ills of a process and develop and apply practical remedies. 


Yet, in spite of the breadth and depth of the Quality Control profession, its 
objective can be stated in simple terms: Satisfactory Quality Assurance at Minimum 
Quality Cost. "Quality Assurance" is here defined as assurance that the customer 
Will be satisfied with the quality of the product; "Quality Cost" as the cost of pro- 
viding that assurance. 


In our quality control activities do we always keep this objective in mind? 


We assure conformance to engineering specifications. Does this necessarily 
assure conformance to customer requirements? 


We administer sampling inspection with risks which can be mathematically 
calculated. But are those risks right for the needs of the business? 


We quite properly survey the quality capabilities of a new, untested vendor. But 
should periodic surveys of his quality control facilities and procedures be continued 
in spite of existing objective evidence of his quality results? Such periodic surveys 
consume many man-hours of highly skilled personnel of both parties. Is this the 
most effective use of those skills? Would it be more effective to devote at least 
part of those many man-hours to, for instance, defect prevention programs? 


We "hold costs down" by offering a limited salary, thereby recruiting an 
ineffective quality control engineer. Would a salary $5000 higher attract a man who 
could effectiveiy help in reducing scrap loss by $100, 000 a year each year? 


The list of such questions could be extended. Some are related to "Quality 
Assurance", some to "Quality Cost", some to both. Unless we provide measures 
of Quality Assurance and Quality Cost, the effects of the decisions we make in 
answer to such questions cannot be evaluated objectively. 


It is not the purpose of this paper to discuss at length various methods of 
measuring Quality Assurance. Suffice it to say that some satisfactory measure must 
be provided, since "Satisfactory Quality Assurance" is vital for the survival of any 
business. Low Quality Cost without Satisfactory Quality Assurance would indeed be 
an attainment signifying nothing. 


Nevertheless, once Satisfactory Quality Assurance has been attained and is 
continuously verified, reduced Quality Cost is, for many businesses, a major 
potential for reduction of manufacturing costs. 


The reason is the rapid increase in quality cost which has been taking place in 
many bus‘nesses for several years and is continuing today. There are many factors 
contributing to this increase. 
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One factor is the increase in product complexity. Because of the interdepen- 
dence of parts and the consequent compounding of proba ilities, the increase in 
quality problems is greater than the increase in the nunber of parts in an assembly. 
Thus as product complexity increases, the ratio of quality cost to total manufacturing 
cost increases. 


Another factor is the increase in precision requirements. We now talk glibly of 
millionths of aninch. We thus press the state of the art, not only in meeting micro- 
scopic precision, but also in measuring adequately to verify the fact. 


Another factor is the increase in reliability requirements. The problem caused 
by this factor is not only the challenge of attaining high levels of reliability, but also 
the enormous cost of test-hours required to prove that it has been attained. 


Another factor is the increase in specification complexity, which presents 
problems above and beyond those of product complexity. Many modern specifications, 
for instance, not only spell out the requirements of the product, but also legislate 
in detail the manufacturing and quality control system under which it will be built. 
And that is only the beginning. The contract calls out several specifications, each 
of which calls out several other specifications, each of which calls out several 
other specifications, and so on, in chain letter fashion. Conformance requires not 
only engineering and manufacturing skill, but a legal mind as well. 


Because of these and other factors, quality cost has become, for many businesses, 
a major part of manufacturing cost. Furthermore, a large part of the quality cost 
may be significantly reducible. Whether such reductions result primarily in price 
reductions or profit increases, the long-term results are bound to improve the 
health of any business. 


* * * 


Quality cost has been here defined as the cost of providing satisfactory quality 
assurance. Such a definition is, however, an over-simplification, and leaves many 
questions unanswered. 


For instance, we have seen "quality cost" fugures which are simply the expenses 
incurred in running the quality control department. As such, they are an attempt to 
measure the expense of doing quality control work. But since the functions covered 
by quality control departments vary widely from company to company, and even 
from plant to plant within companies, the expression "quality control work" remains 
ill-defined. Furthermore, such figures exclude, for instance, the losses due to 
scrap and rework, which are surely part of the cost of providing assurance that the 
customer will be satisfied with the quality of the product he receives. 


On the other hand, figures on "quality loss" have been used somewhat exten- 
sively. These figures measure the saving which would be made if no defects were 
generated. As such, they provide a good measure of progress in a project of limited 
duration and scope. But they hardly provide complete long-term control, for they do 
not include the expense of doing quality control work. 


At Raytheon the following decisions were made: (1) the costs of doing quality 
control work, whether done by the quality control department or any other depart- 
ment or departments, are included in a classification called "quality expenses", 
(2) the costs of all defect-caused losses are included in a classification called 
"quality losses", (3) "quality cost" is the sum of "quality expenses" and "quality 
losses". * 


? 
* See W. J. Masser, "The Quality Manager and Quality Costs". INDUSTRIAL 
QUALITY CONTROL, Vol. XIV, No. 4, October, 1957, Pages 5 - 8. 
The Raytheon system is similar to the General Electric system, except that 
Raytheon's "expense" is the sum of General Electric's "prevention" and 
"appraisal", and Raytheon's "loss" is the same as General Electric's 
"failure". 
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To define "quality control work", Raytheon has written a carefully defined list 
of work elements, reproduced in Appendix A, QUALITY CONTROL WORK ELEMENTS. 
This has provided simplicity in an otherwise difficult area. By means of this it has 
been possible to evaluate "quality expense" with comparative uniformity. 


It has been found desirable to simplify, as far as feasible, the regular periodic 
reports of total quality cost. This is to avoid any significant addition to quality costs 
caused by the work of measuring them. 


For this reason the regular periodic reports contain no breakdowns other than 
the elements required to build them up. Thus, for instance, in the regular periodic 
reports no attempt is made to separate detail inspection from sampling inspection; 
they are both included in "Inspection", which is one of the elements of "Quality 
Expense". If this element is subsequently disclosed to be a significantly reducible 
one, as it frequently is, it can then be broken down into sampling and detailing, or 
into whatever other sub-classifications may seem most promising in the search for 
potentially significant reductions. 


The use of a report form without fine breakdowns for regular periodic reporting 
of total quality cost has proved to be a fortunate breakthrough in simplification and 
consequent universal acceptance of the principle of quality cost reporting. 


The reason lies in the fact that the forty work elements defined in Appendix A, 
QUALITY CONTROL WORK ELEMENTS, are a description of the functions of an 
"idealized" quality control department. If they are the work elements of the actual 
quality cantrol department, then the operating expenses of that department, already 
being reported by regular accounting procedures, constitute the total of the "Quality 
Expenses". Insofar as part or all of some of the work elements are performed by 
other departments, the costs of those work elements are estimated. Then the total 
"Quality Expense" is the sum of the operating expense of the quality control depart- 
ment and the expenses of the work elements performed by the other departments. 


"Quality Loss" is the sum of the defect-caused losses which fall into the 
categories of scrap, rework, downtime, downgrading, and warranty. There is 
usually general agreement that it is worth while to evaluate and report such losses 
anyway, even if the subject of "Quality Cost", as such, had never been brought up. 


The items of the regular periodic quality cost report appear as shown in 
Appendix B, QUALITY COST REPORT FORM. While many of these items are 
self-explanatory, the following clarifying comments are offered: 


2. Adjustments - The purpose of this is to take due account of any 
oddities encountered. For instance, if Safety Engineering is done by the 
Quality Control Department, the expense of this work is credited. Again, 
if some inspection work is charged to no department, but to general over- 
head, the expense of this work is debited. 


4. Special Materials and Equipment - If capital inspection or test 
equipmaat aot Tacladed Wes ie included here. The amortiza- 
tion value is debited for each month. Special treatment is used for capital 
equipment paid for by the customer. 


5. Quality Expénses Other Departments - Here are listed the other 
departments, such as Production, Factory Engineering, etc., which do 
work described in Appendix A, QUALITY CONTROL WORK ELEMENTS. 
The cost of such work performed by each of such departments is estimated 
and debited. 


9. Downgrading - This is the loss due to selling of the product ata 
reduced price because of failure to meet one or more of the product 
specifications. The amount debited is the price difference. 


10. Downtime - This is the cost of idle time of machines, processes 
and operators caused by the fact that the machine or process has been 
making defective product and has been shut down for adjustment or repair. 
It does not include regular maintenance, nor does it include downtime 
caused by lack of good material from a vendor or previous process. 
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l7. Percent 12 to 15 - This is the ratio of Total Internal Quality 
Costs to Cost of Manufacture. It is the measure watched most closely, 
since it reflects accurately the immediately-controllable quality costs. 


18. Percent 13 to 14 - This is the ratio of Warranty Costs to 
Total Quality Costs. This ratio provides some measure of Quality 
Assurance, although it may be greatly influenced by market conditions. 


19. Percent 14to 16 - This is the ratio of Total Quality Costs to 
Sales. It is the ultimate measure of Quality Costs. But it is a statistic 
in which the time factor causes instability. Internal quality costs are 
incurred during the period being reported. Warranty losses and Sales 
which occur in one period are frequently the result of events which 
occurred in some earlier period. 


A periodic quality cost report is issued once each quarter. The figures for each 
quarterly report are the combined results of three sets of monthly figures. 


Thus the data are in form for control chart presentation, in that each group of 
three monthly figures forms a rational subgroup. Therefore the quality cost records 
are kept as control charts. The control limits indicate the maximum fluctuation 
which may be expected to result from the chance combining of the many small 
variables which affect such accounting figures. 


This is a valuable asset, in that it prevents the constant chasing of.rainbows 
which can result from watching the month-to-month fluctuations. The control limits 
in each case indicate the inherent instability of the statistic being charted. Thus the 
control limits for the ratio of Internal Quality Cost to Cost of Manufacture are not 
wide. This points out the relative precision of this statistic as a quarterly control 
figure. On the other hand, while the ratio of Total Quality Cost to Sales has the 
advantage of including Warranty Loss, which definitely must be watched, the statistic 
tends to be "out-of-phase" with Internal Quality Costs, and thus has relatively wide 
control limits. 


It may appear to some that even the ratio of Internal Quality Costs to Cost of 
Manufacture, being published only once each quarter, is a too-long-delayed figure. 
This, however, misses the heart of the Quality Cost Control concept. 


For the long-term control of quality costs is not simply a supplement to the 
day-to-day trouble shooting so characteristic of many existing programs. It is an 
attack on the enduring causes of high quality costs. It is in these enduring causes 
that the major potentials for quality cost reduction are found. 


* * * 


While most of the problems and rewards of conducting a total program of quality 
cost control are yet to be experienced, some characteristic patterns of progress are 
now emerging. A discussion of the patterns may be enlightening. 


At the outset many express surprise that the quality costs are so high, in spite 
of advance estimates which may have been made. Facing up to the actual high 
dollar figures is a new experience. There is a new level of attention to the quality 
cost problem. Questions are asked. Are we doing too much inspecting and testing? 
Are our standards too strict? Are there duplications of effort? Are there too many 
layers of supervision in the quality control department? Is scrap being watched 
closely on a day-to-day basis? 


Admittedly, wrong answers to some of these questions may generate ill-advised 
projects. Some such projects might have the net effect of increasing, not decreasing, 
quality costs. Some might jeopardize quality asqurance. The Quality Control 
Manager must act as a brake on such decisions. But many of the questions do gener- 
ate worth-while projects. These should be pursued to completion. In this way 
initial reductions in quality cost can be made without extensive analysis of quality 
cost figures and without the use of highly specialized managerial, engineering, or 
statistical techniques. 


After a few montis this initial program of projects runs its course. Significant 
reductions in quality cost appear. At best, quality costs then level out at a reduced 
plateau. At worst, they begin to rise again. This may be due to the introduction of 
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new products and the attendant problems, in which case renewed quality cost reduc- 
tion efforts are called for. On the other hand, it may be because the reductions 
were not accompanied by adequate controls. The necessity of installing adequate 
controls.to hold the gains is a constant requirement and the danger of overlooking 
this point is always present. 


After the plateau has been reached, a more careful selection of further cost 
reduction projects is made. This is done by further analysis of quality cost. In 
this analysis the broad classifications of the quarterly reports are broken down into 
finer classifications by product lines, types of defects, cost centers, etc.. From 
among these classifications, further projects for cost reductions are selected on the 
basis of optimum combinations of (1) dollar loss and (2) ease of reduction of loss. 
The specialized techniques of quality control management and quality control engin- 
eering are applied as needed in the pursuit of these projects. Further reductions in 
quality cost are realized. Again the controls required to hold the gains are of the 
utmost importance; otherwise quality costs will climb again. 


After many months a point is reached at which it may seem that little can be done 
to bring about further significant reductions in quality cost. At this point there may 
be, for instance, a few chronic defects still causing large loss, but they are con- 
sidered "state-of-the-art" problems. They are, in fact, tough nuts to crack. Yet 
even many of these problems yield to the advanced techniques of quality control 
engineering, such as the tolerance analysis, the designed experiment, and the 
correlation study. If these advanced techniques are skillfully handled, new infor- 
mation on natural tolerances and on cause-and-effect relations can point the way to 
new breakth~vughs in quality cost reduction. 


* * * 


Thus there is a logical sequence to Quality Cost Control. We start with the 
premise that the objective of quality control is Satisfactory Quality Assurance at 
Minimum Quality Cost. It then becomes apparent that the making of good quality- 
control decisions requires meaningful measures of the attainment of this objective. 
One of the meaningful measures is Quality Cost. Measuring and reporting of Quality 
Cost generates programs of reduction and control of this cost. The gain from this 
reduction and control may be very high, in many cases yielding a profit improve- 
ment comparable to that to be expected from major increases in manufacturing 
plant and sales, or price reductions which will substantially improve the company's 
competitive position. 
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APPENDIX B 


QUALITY COST REPORT FORM 


MONTH 
DEBIT CREDIT NET 


1. Reported Q.C. Department Expenses ‘lil 
2. Adjustments 

3. Adjusted Q.C. Department Expenses 
4. Special Materials and Equipment 


: 5. Quality Expenses Other Departments 


5.1 
5.2 
$.3 


5.4 Total Quality Expenses Other Departments 
6. Total Quality Expenses 


: 15. COST OF MANUFACTURE 
16. SALES 
17. PERCENT 12 to 15 
18. PERCENT 13 to 14 
19. PERCENT 14 to 16 


7. Scrap 
8. Rework 
9. Downgrading 
10. Downtime 
11. Total Internal Quality Losses 
12. TOTAL INTERNAL QUALITY COSTS 

== 
13. Warranty Costs 
14. TOTAL QUALITY COSTS 
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QUALITY AND RELIABILITY MANAGEMENT 


Ralph E. Kuehn, Member of Technical Staff 
Institute for Defense Analyses 


The tremendous cost of unreliable military equipment in a peacetime situation 
and the possible cost of national survival in wartime have caused the inclusion of 
firm reliability requir ts in develop t and procurement documents. That 
portion of the commercial industry working on the research, development and pro- 
duction of systems for the control of continuous processes is rapidly recognizing the 
necessity of providing highly reliable equipment. Reliability has become a topic 
meri:ing the attention of top management so as to arrive at the most effective means 
of planning, organizing, directing, coordinating, and controlling a reliability assur- 
ance program. A reliability program will be suggested starting at the first concept 
of a product and continuing through development, production, and custorner use of a 
product. The discussion will be restricted to the level of a plant with development, 
design, purchasing, manufacturing, and field service capabilities and responsibili- 
ties. 


With reliability becoming prime product characteristic meriting attention from 
the cradle to the grave (first concept to field obsolesence) the combination of Relia- 
bility Engineering with Quality Control in an operating division on a level with Engi- 
neering and Manufacturing has much merit. Figure 1 presents an abbreviated or- 
ganizational chart including a Reliability and Quality Division. 


GENERAL MANAGER 


Engi zering Reliability & Quality Manufacturing 


| 
= 


Reliability Quality 
Engineering Control 


Figure 1 


Advantages of a Reliability and Quality Division include: 


a. Reporting on a rnanagement level where opinions can be given the most 
complete consideration. 


b. Representation on the policy making team. 


c. Separate budget planned at the proper time to assure that effort is ac- 
complished on a timely basis. 
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d. Recognition of the importance of quality and reliability as primary pro- 
duct characteristics. 


The organization for reliability assurance has logically been divided into two 
parts, corresponding roughly to the development and production stages of a product. 
The logical assignment of reliability functions can be partially inferred from Figure 
2. 


Design Reliability 
Va a Product Reliability 


Field Reliability 


Product Life Cycle 


Figure 2 


The design reliability demonstrated by the test of a prototype system built under 
model shop conditions provides a check on the success of the development effort and 
should be a significant factor in the decision to initiate factory production. The re- 
sponsibility for assurance that the reliability of the design turned over to production 
satisfies the customer requirements is a Reliability Engineering task. This must 
not be misconstrued as an encroachment of the designer's responsibility. Engineer- 
ing Design must be responsible and accountable for the design reliability. 


The product reliability demonstrated by controlled tests of systems built in the 
factory with hard tools and production procedures can give a better indication of the 
reliability to be expected when the system is operated by the customer. Careful test 
planning is required to assure this result. Traditionally, Quality Control organiza- 
tions have had the responsibility of assuring that the product agrees with the draw- 
ings and the specifications. It is logical then that Quality Control should have re- 
sponsibility for the reliability of the product shipped from the factory. The differ- 
ence between the design and product reliability is not only a measure of the quality 
of workmanship and effectiveness of Quality Control but also represents the produci- 
bility of the engineering design and the coordination of the Reliability Engineering and 
Quality Control efforts. 


The planning of the reliability program will be discussed on the basis of areas 
of primary interest to Reliability Engineering, to Quality Control and of mutual in- 
terest. The Reliability Engineering planning must start at the earliest possible mo- 
ment in a proposed program. This is necessary as early decisions can set an upper 
limit on system reliability. As an example, consider a complex system requiring a 
central computer which could be analog or digital. The analog computer could be 
implemented by mechanical, electro-mechanical and electronic types. Digital com- 
puter implementations include general purpose, differential analyzer, and functional 
technique types. Using the same rules with regard to component part selection and 
applications relatively crude absolute reliability predictions can be used to establish 
good relative indications of the reliability potential of the various possible computer 
implementations. 
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The development and application of reliability prediction techniques at the block 
diagram, schematic drawing and prototype hardware stages of a system development 
is an essential part of a Reliability Engineering effort. In order to develop, apply, 
measure the effectiveness, and refine the prediction techniques; studies in the follow- 
ing areas should be planned on a continuing basis: 


a. The relationship of component part failure modes and failure rates to 
circuit and system malfunctions. 
b. The effect of the complete environment on the performance of systems, 
circuits and component parts. 


c. The relationship to reliability of redundancy at the component part, cir- 
cuit, and multimode system levels. 


d, The control of reliability tests on component parts, units and systems 
and the statistical analysis of the test results. 


e. The analytical relationship of controlled laboratory reliability tests to 
the field reliability experienced by the customer. 


The implementation of these studies defines the responsibilities of the Reliability 
Analysis group shown in Figure 3. 


The common practice of building a breadboard to prove the feasibility of a con- 
cept can lead to difficulty if reliability has not been considered. Top level scientists 
and engineers can usually get any system to work long enough to demonstrate to the 
sales force and potential customers. Demonstration of new capabilities or signifi- 
cantly higher performance then creates a demand for the system and pressure for a 
drastically short time to production. As a consequence, a production engineered ver- 
sion of a breadboard utilizing untested component parts often goes into production, 
The planning of continuing efforts in the selection and application of component parts 
is an essential reliability effort. 


The selection, application and certification of component parts requires a group 
normally called Component Part Engineering. The selection of cormponent parts re- 
quires a continuous monitoring and evaluation of new component parts as developed 
by parts manufacturers. The selection and application of component parts are 
closely interrelated. The component part engineer should have a knowledge of cir- 
cuitry and expected environment as well as detailed part characteristics. To ensure 
the selection of the best parts and their proper application, work must be planned in 
the following areas: 


a. The preparation of preferred component part lists based on laboratory 
tests and field experience with production systems. 


b. The preparation of specifications establishing the environmental and 
characteristic limitations on the preferred parts. The inclusion of failure rates with 
a demonstration plan is a necessity for high reliability long life systems. 


c. The establishment of a program to standardize on the minimum feasible 
number of different component part types. 


The implementation of the recommendations of the "Ad Hoc Study Group on 
Parts Specification Management for Reliability" should materially improve the com- 
ponent part situation. 
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RELIABILITY 
ENGINEERING 
Reliability Analysis Field Evaluation Component Part 
Engineering 
Environmental Test System Evaluation 
Figure 3 


System Evaluation serves as an unbiased group evaluating the reliability, 
maintainability, operational suitability and specification performance of prototype 
systems. System Evaluation must plan the bench test program so as to get signifi- 
cant, statistically sound measurements of the previously mentioned system charac- 
teristics. The questioning attitude as a method of control must be encouraged if all 
malfunctions are to be treated as foreshadowing future trouble in field usage. This 
is necessary to overcome the natural optimism which tends to view a failure as the 
random one in a billion occurrence that will never happen again. At this stage any 
marginal aspects of the design must receive careful consideration. The maintenance 
of running distribution plots on critical circuit and system measurements can be very 
helpful in uncovering either loose or unrealizable tolerances before production starts. 
It is of extreme importance that the fixes for malfunctions are not of a marginal na- 
ture such as tube selection, matching of comj »nent characteristics and setting of un- 
realistic reduced machining tolerances, These can appear as attractive fixes when 
testing equipment produced in a model shop, but usually become disastrous when re- 
leased to production. It is essential that System Evaluation have the attitude ''Any- 
thing that happens here is sure to happen in the field. " 


Environmental Test can very profitably have all the plant's environmental fa- 
cilities and provide testing services to all areas. Environmental Test performs all 
environmental tests on component parts, assemblies, units and systems. For military 
systems the test conditions are those listed in military specifications modified by ex- 
pected system usage. The importance of testing commercial systems in the environ- 
mental conditions expected in the field often gets too little attention. Life testing of 
component parts, units, and systems under realistic usage conditions is an additional 
function of Environmental Test. The importance of environmental and life tests to 
reliability has had much stress in the literature. This importance, coupled with the 
obvious large expenditures involved, makes test planning mandatory. Test to failure 
and accelerated life testing have received much attention recently. A program 
jointly planned by Reliability Analysis, Component Part Engineering, Environmental 
Test, and System Evaluation is needed, Large savings with a possible increase in 
useful data is possible from a careful combining of life and environmental testing. 
Specific work in the area of correlation of test to failure and accelerated life test 
results with normal laboratory and field experience is needed. The development of 
analytical expressions relating failure rates with test conditions should be the long 
range objectives of such a test and analysis program. 


Dependent on the nature of the product a Field Evaluation group may or may not 
be necessary. The developers of manned airborne systems and automatic chemical 
process control systems as examples have field evaluation groups called Flight Eval- 
uation and Pilot Plant Evaluation. The objective of groups such as this should be the 
evaluation of reliability, maintainability, operational suitability and specified per- 
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formance under controlled conditions approaching those expected when used by the 
customer. In order to attain the objective, efforts must be planned in the following 
areas: 

a. Specification preparation. 

b. Test instrumentation. 

c. Data collection, reduction and analysis. 

d. Operation and maintenance of facilities. 
Attainment of specification performance reliability, maintainability and operational 
suitability in the Field Evaluation situation is normally the signal for production to 
commence. 

Quality Control must plan a program to assure that the product reliability shown 

in Figure 2 approaches the design reliability. Figure 4 shows a suggested organiza- 


tion of Quality Control into three areas, namely, Inspection, Quality Engineering, and 
functional Test and Metrology. 


QUALITY CONTROL 


I : Quality Functional Test 
Engineering and Metrology 
Figure 4 


Inspection has the responsibility of assuring that parts, processes and pur- 
chased items agree with the specifications. This area provides the basic measure- 
ment data which is used for determining the quality and reliability of the parts, pro- 
cesses and manufacturing techniques. The importance of inspection is well recog- 
nized and the methods and procedures utilized have been well documented in the 
litarature. Consideration should be given to introducing the vital reliability factor of 
time into the inspection program. This could be done by instituting periodic life 
tests on parts and processes to determine the time variation of characteristics, de- 
gradations, and catastrophic failures. The feasibility and economy of lot qualifica- 
tion by complete life and environmental testing of a sarnple merits study as a meas- 
ure to assure the continued quality and reliability of a vendor initially qualified dur- 
ing system development. 


Functional Test and Metrology has the responsibility for the testing of assem- 
blies, units, and systems as well as the maintenance and calibration of ali test equip- 
ment and measurement standards. Since most reliability definitions include speci- 
fied or satisfactory performances as a criteria of a reliable system, the accuracy 
and range of the measuring equipment is of extreme importance to quality and relia- 
bility. The planning of the required accuracy of electrical and mechanical standards 
and working test equipment should cover the complete product cycle including re- 
search, development and production, Planned maintenance and calibration checks 
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are needed to assure constant and known errors of all 8 it equip t. Re- 
search in the elimination or compensation for unknown parameters affecting measure- 
ments contributes to the dependability of the measurements. 


The static and dynamic testing of assemblies, units and systems for compliance 
with specifications is a conventional quality measure. Life testing in these areas on a 
sample basis is a definite reliability measure in production. Such tests must be for 
more than the determination of failure rates and satisfaction of reliability specifica- 
tions. It is most essential that all malfunctions be analyzed for cause and that cor- 
rective action be taken on all systematic malfunctions. Production system reliability 
testing requires careful planning in the following areas: 


a. Duration of test. 

b. Failure reporting and analysis. 

c. Definition of failure. 

d. Rejection criteria. 

e. Environmental and operational conditions, 
f. Programmed operational cycle. 

g- Maintenance program. 


Quality Engineering has the responsibility for the planning and implementation 
of methods of prevention, analysis and corrective action in the areas of quality and 
reliability within the manufacturing process. Emphasis on the preventive aspects of 
reliability in manufacturing is a must if high quality and reliability are to be realized 
at a reasonable cost. Experience in a number of organizations has shown that in- 
creased preventive efforts have resulted in reduced inspection costs (without a re- 
duction of detection functions) and a net saving in the total quality expenditures. 


The placement of key Quality Engineering personnel in certain Reliability Engi- 
neering, Development Engineering and Purchasing areas before the new product is 
released for production is an excellent method of taking preventive reliability action. 
The key Quality Engineers in these areas can detect unrealistic tolerances; critical 
part, unit and system characteristics; undesirable vendors and impractical measure- 
ment requirements at an early date. As a consequence, preventive action can be 
taken in development rather than in production. Providing the inspection and test 
areas with complete procedures, instructions, techniques and statistical services is 
an important preventive action. Research in the statistical areas of sampling, de- 
sign of experiments, correlation, and analysis of variance are needed to bring the es- 
sential time characteristic of reliability under better control and measure in the man- 
ufacturing process, 


In order to obtain the data needed for analysis and corrective action, specifica- 
tions and forms for the control of materials, processes, parts and systems must be 
provided. A statistical analysis system for failures in processes, parts, units and 
systems must be set up to cover production. Cordination with maintenance and re- 
pair depots is essential for the analysis of malfunctions and assessment of responsi- 
bility to workmanship, poor design, misapplication of parts, unsatisfactory parts and 
misuse. With the increasing tendency of the prime contractor to purchase a large 
portion of all parts and to subcontract a substantial fraction of assemblies, units and 
subsystems; the maintenance of reliability and quality standards during production at 
vendors and subcontractors b extremely important. 


Quality Engineering should survey prospective vendors and subcontractors for 
their quality procedures, test facilities, reliability programs, and production capa- 
bilities in conjunction with Engineering and Purchasing. After rating and initial se- 
lection, Quality Control must evaluate vendors and subcontractors on the basis of the 
product delivered. Feedback of discrepancy and failure data to the vendor is a neces- 
Sary step in the corrective process. Consideration should be given to having qualified 
and reputable vendors conduct type tests and periodic requalification tests as this can 
result in earlier and more effective corrective action in addition to providing test in- 
formation needed by the prime contractor. 
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Having responsibilities in the production process for specifications, data 
analysis, vendor survey and vendor evaluation, Quality Engineering is in an excellent 
position to be responsible for seeing that corrective action is taken. Coordination 
with the various areas that have to take the necessary corrective action is, of course, 
of great importance. Keeping top management informed by scheduled status reports 
on the corrective action program is than a Quality Engineering task. 


Areas of mutual interest to Reliability Engineering and Quality Control include 
failure reporting, failure analysis and multiple source development on component 
parts and subcontracted items. A reliability and quality program must have a failure 
reporting system if continuity is to be maintained from project to project and if past 
experience is to be most effectively utilized. Failure data is generated on all levels 
from the component part to the system and in all phases from research through de- 
velopment, production, and useful operating life with the customer. It is obvious 
that the primary objective of Reliability Engineering is assurance that reliability is 
inherent to the design released for production. The primary objective of Quality 
Control is assurance that reliability and quality has been built into the production 
systems. Since failure reports properly obtained and analyzed can yield component 
part and circuit failure rates as well as data for evaluating and refining prediction 
techniques, it appears that Reliability Analysis should operate the failure reporting 
system starting with factory acceptance and continuing to obsolesence in the field. 

In the area of in-process failures in inspection, assembly and test, the interest of 
Quality Control is paramount, so Quality Engineering should handle the failure re- 
porting and analysis in these areas. Experience has shown that the assignment of 
people to the field at sample locations with the sole responsibility for reporting 
failures and reliability conditions greatly improved the accuracy and completeness of 
the reports. If reliability predictions is ever to become reliability calculation, con- 
trolled field reporting and reliability measurements are essential requirements. 


During development it is often difficult to justify the development of second or 
multiple sources for component parts and subcontracted items. This often results in 
qualifying additional sources late in the development phase and during the production 
phase. The question of the location of responsibility and authority for development 
and qualification can be a serious one. All too often control is divided or transferred 
during the program. It is felt that this work can be done in Production Engineering, 
Quality Control or Reliability Engineering. In general, each of these areas has sound 
reasons both for doing and not doing this work. Managernent must weigh the pros and 
cons and decide each case on merit, assigning the complete task with responsibility 
and authority from initiation to completion. 


The direction of a reliability program varies from the direction of other engi- 
neering and manufacturing efforts only in degree. While it may be trite to say that 
reliability is everybody's business, the cooperation of everybody is needed if the re- 
liability objective is to be attained. Cooperation on reliability is best obtained when 
the parties involved understand and appreciate the importance of the subject. Hence, 
an active education program requires constant and active direction on the part of the 
Reliability and Quality Division Manager. The use of the following media have proved 
of value in reliability education: 


a. Manager training program. 

b. On the job training. 

c. Engineering seminars. 

d. Attendance at conferences and symposiums. 
e. Posters and company newspapers. 


The scheduling of reliability tasks is of extreme importance and requires de- 
tail direction. As an example, life and environmental testing at the component part, 
circuit, and unit level must be completed at an early date if test results are to be 
used to refine the reliability predictions. Availability of test data revealing unsatis- 
factory component parts and design at a sufficiently early date permits corrective 
action or paraliel developments so as to prevent more serious difficulties in produc- 
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tion. A careful balancing of available test time and sample size (and hence economic 
considerations) must be made if scheduling is to have proper direction. 


Coordination of the total reliability effort is a sensitive area as reliability and 
quality depend so heavily on design and manufacturing techniques. A Reliability and 
Quality division can most effectively get its recommendations accepted by selling and 
persuading the responsible areas based on the self-interests of the area. Establish- 
ing a record of competence and sound judgment in the various Quality and Reliability 
services does much to develop the confidence on which coordination is based. It must 
always be remembered that a Reliability and Quality division is essentially a service 
organization and that the responsibility and accountability for design reliability and 
product quality reside in Development Engineering and Manufacturing respectively. 


Reliability Engineering has coordination problems among its several areas. 
As an example, Reliability Analysis, Component Part Engineering and Environmen- 
tal Test must coordinate the life and environmental testing of component parts. Re- 
liability Analysis is concerned with failure rates, degradation, statistical signifi- 
cance, amd modes of failure. Component part Engineering is interested in estab- 
lishing application limits and obtaining certification data. Environmental Test is 
concerned with instrumentation requirements, range of life and environmental test 
conditions, frequency of measurement, and automated testing. These requirements 
of the several areas are sometimes in conflict and coordination is needed to assure 
the greatest benefit to the reliability effort. 


Reliability Engineering must coordinate with Development Engineering, Pro- 
duction Engineering, Quality Control and Research in the following areas: 


a. Scope of standardization effort. 

b. Selection and qualification of second sources. 
c. Form and content of failure data reports. 

d. Design methods for reliability. 

e. Requalification of vendors. 

f. Reliability efforts on research systems. 


These are only a few examples from a lengthy list and are chosen only for illustra- 
tive purposes. 


Reliability Engineering and Quality Control have coordination problems with the 
customer. Reliability Engineering needs field failure data which must be provided by 
the customer or collected on his premises, The natural desire of the manufacturer 
and customer with high stakes in equipment reliability to both collect data and ana- 
lyze it in their own way requires a high management level of coordination to avoid 
duplication and ensure a reasonable agreement between reported results. Quality 
Control feels that first hand verification of field discrepancies is necessary for ex- 
pedited corrective action. Hence, they desire representation on the customer pre- 
mises rather than being dependent on second-hand information that is often difficult 
to confirm independently. 


Having set up specifications and procedures to ensure in the house reliability 
and quality, coordination with vendors and subcontractors is a necessity. The dif- 
ferences in organizational structure, procedures and management philosophies pre- 
sent many problems. Many compromises are normally necessary to arrive at mu- 
tually satisfactory quality and reliability programs at vendors and subcontractor 
facilities. The advisability of reconciling differences and spelling out commitments 
in the contract negotiation and purchase order phases are obvious. For the manufac- 
turer with prime responsibility for a system, careful aitention and consideration in 
this area can eliminate a multitude of troubles. 


Quality Control often has coordination problems with Production Engineering 
and Manufacturing. The assignment of priority for corrective action based on the 
correlation of in-process and field failures is such an item. The definition of re- 
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sponsibility for the chree functions in the product improvement program requires 
continuous coordination. The determination and reporting on the effectiveness of 
changes in manufacturing techniques, inspection sampling, acceptance specifications, 
and design changes as these changes are broken into production are Quality Control 
functions that should improve coordination efforts. 


The preventive nature of a Reliability and Quality program increases the criti- 
cal nature of the control function for management. Only by establishing sensitive con- 
trol measures car the necessary changes be made in the reliability effort before the 
design is committed to production. The use of time scheduling with frequent reports 
on the progress of the various reliability tasks is an excellent basis for reassigning 
manpower and facilities to assure compliance with the schedule. The use of control 
techniques such as PERT and PEP can be of great value in meeting the reliability 
schedule. 


A major control for the technical evaluation of the Reliability Engineering effort 
is the setting of mean-time-to-failure goals for units, subsystems and systems on a 
time schedule corresponding to the planned reliability prediction, design and meas- 
urement check points in the program. At each measurement time, control may be 
exercised by modifying predictions and changing individual unit and/or subsystem 
mean-time-to-failure goals so as to realize the system reliability goal. If adjust- 
ment is not possible within the subsystem reliability apportionment, redesign action 
may have to be initiated or a product improvement program planned if schedules 
don't permit redesign. 


Frequent and scheduled reports on the release of component parts, results of 
life and environmental tests of component parts, failure analyses on unit life tests 
and vendor quality evaluations provide excellent controls for a component part stan- 
dardization program. Testing of additional component part types within a category 
may be cancelled as a result of a certification report or additional testing may be 
initiated as a result of discouraging test results. Careful utilization of these controls 
can assure the certification of reliable component parts before the system is released 
to production, 


In summary, it can be stated that the quality and reliability effort must start 
with the first concept and continue through development, design, and production to 
ensure high reliability in operational systems. A Reliability and Quality division pro- 
vides the continuity of services needed to assure attention to all details at the proper 
time in the program. The exercise of skill in the management functions of planning, 
organizing, directing, coordinating and controlling is necessary if the reliability and 
quality objectives are to be realized. 
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TOTAL QUALITY CONTROL IN SMALLER INDUSTRIES 


Allin P. Deacon 
Deacon & Associates, Brantford, Canada 


INTRODUCTION 


In his paper “The Challenge of Total Quality Control", Dr. As V. Feigenbaum of the 
General Electric Company has described total quality control as the integration of 
quality control efforts from market research and engineering design through to ship- 
ment and final installation of the product such that the customer remains satisfied and 
the costs of maintaining that goal are at a minimum. He has pointed out that welding 
together the quality efforts of all the functions in a company pays off in lower 


quality costs and better interdepartmental relations than c+. be achieved in any other 
mannere 


Dr. Feigenbaum has pointed out further that the cost of controlling the quality 
shipped and the cost of satisfying the customer can be minimized only if attention is 
paid to the prevention of defects. Increase in cost in this area will reduce the cost 


of appraising quality and replacing or repairing defective goods in the plant or at the 
customer's location. 


It is true that the General Electric Company is a very large organization. There 
are many in small industries who will say that, "Yes this is worth while for large com- 
panies which have money to spend on elaborate control systems, but does it really have 
its place in small industries? Is the cost of setting up such a program really worth 
while? Will it pay off?" In this paper I propose to describe how this concept of total 
quality control can and has saved dollars for small companies and given management and 
employees a new concept of co-operation in achieving customer satisfaction with minimum 
interdepartmental conflict. 


WHAT DOES THE CUSTOMER WANT 

How do we go about achieving total quality control? First of all we must learn 
what our customer wants. Sometimes the only way to determine this is by studying com- 
plaints to learn what he does not want. If we are starting with a new product to be 
made to a set of specifications provided by our customer we are in a fortunate position. 
It enables us to sit down with our sales department and the customer and go over the 
Specifications concentrating on those details which have raised any doubt in our mind 
as to his requirements and learning from him in particular what quality characteristics 
are of prime importance to him in the function of the product or its assembly into its 
final location. As Dre Feigenbaum has indicated in his paper, a great deal of grief 
can be prevented if we make sure what is required to make our product function correctly 
after it has been received by our customer and installed. The sales organization is one 
of the keys to good communications with our customers. 


QUTGOING QUALITY CONTROL 

Once we are sure what is required, our next step is to make sure as well as we can 
how well the product conforms to the customer's requirements or our engineering design 
before it leaves our plant. Final inspection is supposed to achieve this purpose. 
However, experience has shown that if quality is not controlled during manufacture some 
defective products will reach our customer no matter how much inspection is applied 
just before the product leaves the plant. Final inspection can be extremely costly and 
yet not entirely reliable. Final inspection must be considered only as a reporting of 
quality achieved during design and manufacture in comparison with the requirements of 
our customers. 


PROCESS QUALITY CONTROL 
If 100% final inspection is not 100% effective while at the same time causing high 
cost in rejects, what can we do? In-process inspection will give us early warning of 
trouble and a chance to correct the process. This can be established in either of two 
waySe 


The first is to establish inspection stations at various stages of manufacture, 
using various sampling plans and 100% inspection when necessary to catch defectives. 
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The second method is to set up patrol inspection in an effort to discover operations 
which are producing defectives before much product is manufactured. In either method 
there are two prime requisites. The first is that the operator is given whatever mea- 
suring equipment is necessary to enable him to measure the quality of his own operatim 
The second is that the results of any inspection from his finished product must be 
given to him immediately. Beyond this, foremen and supervisors must be kept aware of 
any tendency of operators to put off making necessary correction to produce a good 
product. 


Defects must be prevented. The whole objective of patrol inspection or sampling 
inspection must be to measure the effectiveness of the control of quality by the oper- 
ator and report to him or her and the supervisor what sort of quality performance is 
being achieved. Detailed information regarding the kind of defects can help the oper=- 
ator or supervisor decide what corrective action needs to be taken. 


There are instances when the control of quality is somewhat out of the hands of 
the operators. Such a condition can arise when the material is off standard, or the 
equipment being used in the manufacturing process is not capable of producing within 
the specified tolerances, or the measuring equipment being used is not sufficiently 
accurate. Knowledge of quality of the material being used can be gained through pro- 
per sampling inspection as the material is received. Costs due to defective material 
can be avoided by the manufacturer if he knows of the condition of the material early 
enough to either get a replacement shipment from his supplier or make arrangements 
with the supplier to pay for extra operations required to produce a satisfactory 
product. ’ 


CAPABILITY FOR QUALITY 

The capability of the equipment being used in the manufacturing process is one of 
the most important factors in the control of quality. Very few operators are skilled 
enough to produce to specification when the equipment he is trying to control has a 
natural variation which exceeds the tolerances allowed. I have seen a skilled oper- 
ator produce within the specification limits while a study was being made of his oper- 
ation but yet the product reaching the assembly line had a considerable percentage 
outside the specification limits. Therefore, it is worth while to have capability 
studies made of all equipment, particularly that which is producing defective product, 
in order to determine whether it will produce to specification or must be repaired or 
replaced. 


The accuracy of the measuring equipment is again extremely important in the con- 
trol of quality. It is unfair to ask an operator on a grinding machine to control 
parts to a total tolerance .0005 of an inch and provide him only with a snap gauge set 
to the upper and lower limits. First of all, in setting the snap gauge, the amount of 
pressure used by the inspector may be different than that used by the operator when he 
receives the gauge. If he is measuring a round piece with a gauge which has been set 
up with standard gauge blocks, it is unlikely that his measurements will correspond 
exactly with those gauge blocks. Second, he can only tell when the parts he is grind- 
ing are at the upper or lower limits; he cannot tell when he is approaching either 
limit and therefore cannot prevent parts from being made at or outside the limits. 

The alternative of course, is to provide him with a sensitive gauge which will measure 
the diameter exactly in increments of .0001 or better still .00005. 


An important part of in-process control is gauge control. All gauges whether the 
property of the employee or the company should be inspected regularly for accuracy 
sufficient for their purposes. 


INCOMING QUALITY CONTROL 

Receiving inspection is necessary if final quality is to be controlled. Whether 
the material is to be used in our own manufacture, or consists of components which are 
packaged for resale, we must be sure that the quality meets the customer's require- 
ments. Let us not make the mistake of assuming that because we did not make the mat- 
erial we are not responsible for it. Let us remember that the customer considers us 
completely responsible for all materials received from uS. A well planned program of 
receiving inspection using statistical sampling plans will keep us informed of our 
vendors' quality. 


Receiving inspection does not end with knowledge of quality or return of defective 
shipment to our vendors. If we are to control quality at the lowest cost we must work 
with our vendors to help them improve their quality. Let us remember that if a vendor 
is having trouble and has to inspect his product 100%, the cost of his own quality 
control will be passed along to us in the price of the goods. Therefore, it is impor- 
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tant that we keep him informed of the mistakes which he makes and whenever possible 
teach him some simple quality control techniques which will help him control quality 
during his manufacture, and insure us of good quality without costly sorting, rework 
and replacement. 


DESIGN QUALITY CONTROL 

Up to this point we have considered quality control in the manufacturing process 
under conditions where no consideration is given to altering specifications. This is 
the usual attitude taken by the sub-contractor manufacturing for a customer who has 
provided his own specification. The only time that a sub-contractor is likely to seek 
a specification change is after he has found that he is unable to adhere to that spec- 
ification. Interestingly enough this also happens in larger companies where good 
liaison is not habitual between the manufacturing department and the engineering de- 
partment. 


In a program of total quality control close liaison between engineering, quality 
control, production engineering, and manufacturing, is an essential ingredient for 
success. Thus the manufacturer who is producing to customer specification is well 
advised to pass on te his customer not only his problems in the control of quality due 
to close tolerances but also the cases where he can control quality easily with closer 
tolerances than specified. It could be very much to the advantage of the customer if 
he can reduce the tolerances of that part and allow wider tolerances on another. Such 
co-operation between you and your customer can only improve relations to the extent 
that your company is preferred over otherse Any Savings which a vendor can pass on to 
his customer not only reduces the cost of the customer's finished product but also 
invites new business. 


QUALITY OF SERVICE 

Finally let us consider the service we give to our customer after the goods are 
delivered. Let us be honest from the beginning and discuss with the customer the 
possibility that some defective product will reach him in spite of our quality control. 
While we will do our utmost to prevent the shipment of defectives something may happen 
during manufacture which will produce a low percentage of defectives unknown to us. 
It can happen particularly with short runs. Therefore let us enter into an agreement 
with our customer regarding the acceptable quality level for our product, that is, the 
percent defective which he can afford to accept at a given price level. If his use of 
the sampling plans for that acceptable quality level results in a rejection of our pro- 
duct we must be prepared to do whatever is necessary to sort the defectives out of the 
lot and make any necessary replacements. It is at this stage that the importance of 
control of quality during manufacture becomes most evident because only with it can 
serious delays be minimized and relations with our customers kept sweete 


A CASE HISTORY 

It has been my fortunate experience to work with a number of small manufacturers 
who must practice total quality control in order to increase their business in the face 
of the keen competitive conditions of today's markets. One of these, the Turnbull 
Elevator Company Limited, Toronto, Ontario, is somewhat unique in that the product is 
designed to the customer's special requirements, produced in the company’s plant in 
Toronto, and erected in buildings in every province of Canadae Their control over 
quality extends from the design stage right through to the final installation and their 
products are competing successfully with larger manufacturers. 


The installation of quality control began almost a year and a half ago with a 
study of defects originating in each department being made to find the most common type 
of defects, the completeness of information provided in specifications, the adequacy of 
tooling and gauges, and which products appeared to be causing the greatest difficulty 
and highest costs. At the same time a meeting was held with construction supervisors 
to encourage the flow of information from the erecting crews back to the factory re- 
garding defective material and shortages reaching them. These reports gave us a means 
of measuring the effectiveness of the control measures we were introducing. 


In this type of manufacture very few items are ordered in large quantities. There- 
fore, attention was focused on the control of quality of each operation, and provision 
of some incentive to each department to improve control of quality. This was achieved 
through departmental percent defective charts, placed on display and plotted on a 
weekly basis. As a result of operator interest and management action to correct the 
conditions which were found to contribute towards defectives, the percent defective 
showing on these charts declined in a very satisfactory manner. 
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It is interesting to note that an increase in inspection force was not necessary. 
An analysis of their duties revealed that they were performing extra duties belonging 
to other functions, and 100% inspection on some products. Ome such product was made 
in large quantities for use in the electrical controls. A study of the process for 
these parts resulted in the development of new toolage which improved it to the extent 
that sampling at regular intervals was sufficient to control the quality. In this case 
quality has been improved, production increased, and therefore, manufacturing costs re- 
duced. In addition, inspectors now have more time to do the important job of measuring 
the quality being produced. 


Capability studies have been made on equipment that was producing defective pro- 
duct. Many tools, dies, and fixtures have been investigated when the parts did not 
agree with the specification. In many instances it was found that the difference be- 
tween the finished part and the drawings were changed by the engineering department. 
In other cases changes in the drawings were requested to facilitate manufacture and 
assembly. 


A major problem was the application of tolerances to old drawingse As in many 
industries of this size which have been in existence for many years, a large number of 
drawings in the files did not show any tolerances. Without tolerances it was difficult 
for an inspector to judge whether a part was acceptable or defective. The procedure we 
followed was to propose to the shop nominal tolerances for fractional and decimal dim- 
ensions for each type of operation. After receiving their agreement we submitted these 
tolerances to engineering. After a final settlement was reached, instructions were 
issued to the shop to work to these tolerances except where tolerances and fits were 
specified on the part drawing. 


In this plant as in many shops this size, much of the measuring equipment is owned 
by the operators themselves. Some of this equipment was found to be something less 
than perfect. As a result, a survey has been made of all the measuring equipment in 
the shop, listing its location and owner, and McBee Cards have been made up for each 
item. In future all equipment will be examined and checked on a regular basis, using 
the McBee Card File to designate what equipment is to be checked each weeke 


The control over punch press dies is very simple. Whenever an order is completed 
the last piece off the die is sent to the tool room with the diee The piece is checked 
for dimensions and finish with special attention to burrs, and the die reconditioned if 
necessary. No die is put back in storage without an OK tag from the tool room or an OK 
by the inspector. 


Some special studies have been instituted on items where the cost of manufacture 
are high or the reliability is of extreme importance. These studies are directed to- 
wards finding what factors have the greatest affect on the quality of the article, 
either in the finished state or at each operation. The results of these studies will 
be a change in the method of manufacture to reduce the cost wherever possible and im- 
prove the reliability where necessary. One item in particular, a resistance panel, is 
made from an asbestos board which is very difficult to cut. Preliminary studies have 
indicated that another method of cutting will reduce scrap considerably below the pre- 
sent 18% without increasing the labor cost appreciably. 


In another instance the component had to be adjusted after mounting. If it fails 
it can cause a serious loss of time and great inconvenience to the customer. Up to the 
present time the studies have resulted in improved toolage and improved quality on one 
part, and also an indication that an important factor in the operation of the component 
was being disturbed by the adjustment that was made after assembly. 


In the receiving department, sampling inspection is applied to all purchased com- 
ponents in accordance with their importance. The sampling plans have been designed to 
minimize the amount of inspection to give the necessary protection. Where trouble is 
found, suppliers are being instructed regarding the defects, and a start made in sug- 
gesting simple quality control procedures. 


One of the complaints from the erectors in the field was a shortage of parts in 
the shipment. A study was made by sampling the shipments and the causes of the short- 
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ages traced to their sources. As a result the procedures relating to shipment of mat- 
erials from stock and from the manufacturing floor have been revised. All those con=- 
cerned have become very much more conscientious in making up the shipments. 


Because of the construction of elevators which must be shipped in the “knocked 
down" state, and in shipments which can be several weeks apart, the effort to control 
quality is concentrated in the design and manufacturing stages. The final inspectors 
are the erectors and the field adjuster who makes the final test on the completed 
installation. 


The result of all this work has been a marked reduction in the complaints from the 
field with regard to both quality and shortages. Some valuable changes have been made 
to reduce costs of manufacture. Each department in the company has b much more 
aware of its role in producing the quality which will satisfy the customer with less 
installation time, and less delaye In addition, there is a constant feed-back of in- 
formation on manufacturing end design quality with continual improvements being made. 


CONCLUSION 
Nine basic principles will assure us of total quality control and a program of 
prevention which makes substantial savings for our companys 


(1) There must be full knowledge of what the customer requires. 

(2) The specifications must be clear and have realistic tolerances. 

(3) The operators must have equipment which is capable of producing *o these 
tolerances. 

(4) The operators must have measuring equipment which can tell them when they are 
within specification and when they are not. 

(5) Incoming materials must be inspected unless it is known that they are not likely 
to present a problem in our manufacture. 

(6) We must maintain good relations with our vendors and help them establish their 
own control of quality. 

(7) We should be ready to ask for and make changes which will improve the product for 
our customer and reduce its coste 

(8) We must carry on enough inspection on outgoing products to know that we are con- 
trolling quality satisfactorily. 

(9) Our quality control department must be managed in such a way that it co-ordinates 


effectively the efforts of every one in the company towards producing good quality 
and satisfied customers. 
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A CHALLENGE TO LABOR AND MANAGEMENT 


4xel U, Sternlof, Division Q. C. Engineer, Retired 
American Steel and Wire Div. U, S, Steel Corporation 


Labor Unions have accomplished a great deal of good for the people of America, 
They have enabled the so-called "down-trodden" laborer to earn the highest of wages 
ever in the world's history, They have contributed no small amount to the comfortebie 
living that is ours to enjoy today, They have been highly instrumental in improving 
the working conditions under which man has labored for many years, They have obtained 
for us social legislation that eases the minds of our citisens who are approaching the 
retirement age. Yes, organized labor has actually taken a very important part in af- 
fording the American people the highest standards of living in the entire wrld, 


With these remarks as a sort of introduction, I a not wish to convey the idea 
that all these accomplishments are due solely to the efforts of organized labor; they 
have, however, been greatly instrumental in obtaining them for us, But, up to this 
moment, have we ever asked ourselves this question; What has been the cost? What is 
the price that we are paying for all these gains? Have we been blind in our hasty pur- 
suit of the dollar and all that it will bring us? I will right now admit that I have 
far mre hind-sight than I have fore-sight, but then haven't we all the same? Admit- 
ting this, let us then become introspective and self-examining. Above all else, let us 
be brave enough to admit our mistakes of the past and let us all have the "grit" and 
the "guts" to correct them, or at least let us see that we do not make the same mis- 
takes again, 


Industrial Management today finds itself in a very precarious position especially 
as related to the situation between organized labor and industry. We mst remember 
that this relationship in general is set or determined by the so-called "big fellows"; 
the small businesses mst follow or cease to exist. Agsin management must plead guilty 
in the situation in which we find ourselves because it has repeatedly yielded to labor 
in giving "an inch" only to find that in the end labor has taken the proverbial mile, 
Finally management wakes up to the fact that it has lost control of the plant and ite 
operations, 


The plant supervisors who, we all know, are on the scene at the very heart of the 
operations and who are familiar with all the details, will render their opinions in the 
early stages or steps of a grievance only to have the Industrial Relations departmat 
refuse to back them up and management yields to labor in order to save the bother of 
going to the next higher step in their grievance procedures, So it is in this manner 
that we of management give that inch only to lose the mile, In the last strike of 
labor against Big-Steel, did we not see that the industry wanted labor to grant a few 
concessions, only to be told where they could go, By stretching our imaginations mere- 
ly a trifle, we could say that the industry wanted to get back some of the past 
"inches" that they had previously given away. 


At the close of each strike when the generals of both parties involved sit dow to 
bargain, to settle their differences and to write up @ new labor contract, we hear that 
the concessions granted by industry will be more than made up by the increased product- 
ivity on the part of labor, But how many of us, to 
these promises materialize? I have seen plants in which bo 
laborers will admit that they get 8 hours pay for 5 to 6 ho 
sult? In this country, with its high taxes, so many of them deeply hidden but sorely 
felt, our native industry cannot compete under such conditions against foreign produ- 
cers, In other words, we are losing our markets, 


The members of the Metals Technical Committee of the A S Q C will recall that it 
was not so long ago that we heard that Japan was shipping steel rods into this country 
of far better quality and made to better tolerance limits at lower costs than we could 
possibly meet, Then, too, let us look at the compact-car situation, This market, long 
neglected by our native industry, was lost to us with its subsequent great steel tonn- 
age. We can go on and on with similar illustrations, The big $6) question is, What 
are we going to do about it? It is high time that the three groups that should be vit- 
ally concerned about this situation should get together to think and talk this matter 
over, The three groups, of course, are Management, Labor and Government, These three 
must wrk more closely together than ever before in our history in order to held our 
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present markets and in order to gain new ones, My friends, this situation, whether you 
realize it or not, is very serious; our future industrial welfare might well be depend- 
ent on what we do about it. My challenge to industry, labor and government, in this 
case is to forget your differences (if there are any) and strive for greater product- 
ivity and this productivity MUST be of better quality than ever before, 


Unfortunately, quality of product in the past has been mre or less haphazard in 
a great many of our industries. In several instances we have seen rejects or inferior 
products "slipped in" with good materials trusting to luck that they would not be 
caught. Inspection was considered and looked upon as a necessary evil and was always 
the first department to be cut back at the least provocation, If business dropped off 
to any great degree, that department was sure to be practically wiped out, And once 
cut back it was always mst difficult to get the men back on the job. 


Not so long ago (and even in our present day) it was considered good practice to 
take a certain fixed percentage of a lot in order to check its qualities and often a 
second sample would be taken if the results of the first were unfavorable, If the sec- 
ond test came up to specifications, the results of the first test would be promptly 
forgotten, With the advent of S, Q. C., industry hopped on the band-wagon because man- 
agement believed that with this new tool, all their inspection troubles were over; it 
was a veritable panacea for their industrial ills; inspection costs would be sharply 
curtailed; no more reworks and scrap; etc., etc. In fact mst of the plants had very 
little if any inspection costs, 


I had the pleasure recently to be called by a small industrial establishment that 
has been engaged in the manufacture of textile machinery for three generations and were 
now having some serious commlaints on quality of product. I was astonished to learn 
that all their employees were old men who had been in their employ for 25 to 50 years; 
they knew their jobs, the product, and were faithful and trusted employees, I was told 
that when the product left their hands, it was perfect. I requested permission to look 
over their past inspection records and I was very much surprised to learn that there 
were none, I can not say anything about the West, but this situation is quite common 
in the eastern part of our country and especially is it true relative to New England, 
Industry here seems to trust to luck and frequently they run into trouble. Consequent- 
ly, when these industries board the S Q C band-wagon for a solution to their troubles, 
they are mst impatient and want immediate results before they spend a cent. 


I have seen many industries that sail along from day to day with 12 to 15 per cent 
scrap and never bat an eye lash; but, let it hit 25% for one day and the whole manage- 
ment staff runs amuck, You and I know that with 12% fraction defective that we will 
hit 25% on some occasions, If that same energy on the part of management were exerted 
on the 12 to 15 per cent to bring it down to some lower level, say 5%, a great deal of 
money could be saved, I have in mind an industry of this type. This concern has been 
in business for over 100 years manufacturing, in the last 25 years, a product that has 
been in steady demand in large quantities with the government as the prime customer, 
With the signing of a new labor contract, they suddenly realized, after six mnth's 
study, that they could not meet competitive prices. They hired a good quality control 
engineer and assured him that he would have full backing as the situation was desper- 
ate. 


The engineer spent one whole week just getting acquainted with the men, the mach- 
ines, the materials and the processes used, The following week, he inquired about past 
inspection records and was surprised at the voluminous amounts of historical data that 
was available in files, desk drawers, and even packed on the shelves in the so-called 
process control office, None of this data was filed in any chronological order, but 
was simply tucked away by the inspector at the end of his shift. After a few days of 
digging, the results of the last two month's production were taken just to see what the 
history was. 


In Figure 1 we have the result as shown by a simple frequency distribution which 
covered the two shifts of operation, Figures 2 and 3 show the same data, but here we 
see the separation of the tw shifts. Nowhere on the data sheets could any reference 
be found in regard to the specifications, All of the measurements were made to the 
nearest ten-thousandths of an inch, Quality Reports were then filled out as is indi- 
cated in Figures h, 5, and 6 showing respectively that, if the processes were operat- 
ing in control, the per cent of product beyond specifications would be approximately 
42%, 21% and 73%. Up to this point, the engineer's task had been fairly easy, His 
next task is to have his first session with top brass, Here again he was amazed to 
learn that top management had been aware for some time of the large number of rewrks 
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FIGURE 1 - - Frequency Distribution -- Combined Data 


and reprocess material or pieces going through daily, but now the answer was "Now we've 
got to do something about this", 


Acting on the advice of the engineer, the union grievance man was called into the | 
session and the entire situation was explained step by step with the request from man- P 
agement that the union get in back of the quality control program as proposed in order te 
to wipe out the high amount of rewrks so that the competitive price could be met with- ele 
out the cutting of wages or the loss of the business, Here it was extremely fortunate ay 
that the grievance man was one of those rare individuals who had a good head on his ; 
shoulders with both feet on the ground, 


The remedy suggested by the engineer was as follows: : 


1. An educational program for the supervisors and the operators involved, 


2. The institution of a sampling plan and the careful measuring of the product 
by the operator who would plot the point on the chart. 


3. A follow-up by the quality control observer or inspector. 
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FIGURE 2 — 7 = 3 Shift FIGURE 3 -- 3 ~ 12 Shift 


Let us now look at the Figure 7 for the results achieved after the adoption of the 
suggestions, Everybody from the manager down was very well satisfied with the outcome. 
After a month or two under the sampling plan, management decided to discontinue this 
inspection station now that the process had settled down only to learn the hard way 
that it was necessary to go back to interval policing. Human nature being what it is, 
men will tend *o get careless when not under surveillance. 


New England industry is very slow to grasp the full possibilities of this new tool. 
I sometimes wonder if we as potential salesmen of quality control are not doing too 
much to publicize the n-th degrees of the science such as Operations Research, Relia- 
bility, Quality Assurance, etc, How fine shall we cut this piece of steak? Please do 
not misunderstand me, Each of these ramifications has its place in the highly techni- 
cal field of mdern industry, but why do we not get our fundamental processes under a 
state of satisfactory control first. We can, you know, be pemny wise and pound 
foolish. 


May I summarize that which I have tried to say up to this point, American indus- 
try and the American people face a very serious situation that threatens our economical 
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life. Industry mst adopt a hard and fast policy with labor and mst have a strong man 
at the head of its Industrial Relations department. Do not give in at the very first 
step of a grievance on the theory that it is better to give "this inch" in order to 
save the bother and the expense of going to the higher steps. As I said earlier, you 
give this inch and you will lose the mile, Sometime, dy, be it in 
Labor, Industry, or Government must be big enough to say a "STOP to this ever-increas- 
ing spiralling of wages permitting foreign industry to come in and take away our 
markets. 


Second; Management and Labor must cooperate on the installation of and the main- 
tenance of quality control, Labor mst realize that quality control decisions are 
based upon fact and not upon guesswork, We are in the position today of attempting to 
compete with foreign industry possessing the most modern equipment and the latest tech- 
niques with the lowest of wages, 


Finally: It is my confirmed opinion that it is of the utmost importance to get our 
fundamental processes in a state of satisfactory statistical control and then MAINTAIN 
that contrel through police action before we begin to further confuse management with 
the deeper and more highly technical aspects of this science, 


Percent 
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Percent 
Below Above 
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| 
0,00 72.91 
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27.09 


FIGURE 6 --3 to 11 Shift 
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BREADTH ©R DE)TH? 
Training for Quality Control and Operations Research 


Chester H. McCall, Jr., Project Director 
Booz, Allen Applied Research, Inc. 
Bethesda 14, Maryland 


It is surprising to many quality control people when such a subject as training 
for quality control is broadened to include a discussion of training for operations 
research, However, as far back as 1951 it was recognized that quality control was not 
the only answer to industry's search for better ways to operate. In an intriguing 


article! 3) 


it was noted that 

“The over-all problem confronting industrial management from the incep- 
tion of a product to its completed manufacture and distribution is the prob- 
lem of Industrial Operations Research." 


Littauer goes on to state later in the article that... 


"In problems of operations research, whether civilian or military, 
statistical quality control in its most general sense as the means for 
analysis of causes is indispensable." i 


This statement would seem to imply that statistical quality control is a part of opera- 
tions research. Unquestionably this philosophy is not universally accepted. 


As early as 195362) it was suggested that perhaps operations research was only 
the application of statistics and common sense to problems of business. Or possibly 
it was just a comprehensive term for existing activities such as market research, 
quality control, or industrial engineering. Here again a tie between QC and OR is 
noted. 


Two years later the relationship of operations research to quality control and 


vice versa was somewhat clarified in two articles by Dr. Paul oinsteaa*)(5) | presently 
ASQC Chairman of the Operations Research Committee. However, in a panel discussion on 
"Operations Research in Quality Control and Quality Control in Operations Research" 
before the Washington Section in March 1960, it was evident to the audience and par- 
ticipants alike that it was still not clear where the two disciplines differed and 
where they were similar. 


This paper will not attempt to settle any of the possibly conflicting viewpoints 
on OR and QC. Rather, the theme will be to discuss the training problems of these two 
fields and to point out similarities and differences. 


WHY TRAIN? 
The basic philosophy on training held by this author and many others is best 


stated in the following extract from seference'* ) : 
"The continuing develcpment of any intellectual discipline, however, 


cannot be guaranteed without the education and training of a steady stream 
of persons prepared to carry on and extend its traditions." 
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This paper assumes that training is vital and that the development of ‘sound 


training programs will continue to receive the attention of those in industry, govern- 


ment, and the universities. 


FORMAL TRAINING 


There should be no question in the minds of those in quality control or operations 


recearch which of these two disciplines has progressed further in its development of 
formal training programs. The Operations Research Society of America commenced its 
Journal in 1953. The American Society for Quality Control started Industrial Quality 
Control in 1944. Although ORSA began publicizing its research nine years after IQC 


appeared, the college programs leading to degrees in Operations Research far outnumber 


those available in Quality Control. Does this relationship hold in the same way that 
one is more likely to find a program leading to a degree in American History, rether 
than a degree in History of the War of 18127 


As a statistician, I am unwilling to accept the thesis that quality control is 
necessarily a part of operations research, At the same time I cannot state that 
operations research is a part of quality control. Why then does operations research 


seem to be leading the way in formal training? One reason is undoubtedly the cramatic 


success OR practitioners have had in introducing these techniques in military and in- 
dustrial problem areas. The pay-off for good quality control procedures is equally 
imposing. However, these pay-offs have frequently not been recognized as the result 
of applications of quality control methods. 


A further point here is that OR tends to deal with total models while QC has 


unfortunately often restricted itself to minor parts of the operations of an industrial 
plant. It is these points plus a more eggressive group of scientists which have caused 
greater numbers of our young talent to seek formal training in operations research than 


in quality control. 


The tendency in university training for OR is to prepare a man to handle a broad 
spectrum of classical problems in such areas as queueing, model building, linear pro- 
gremming, game theory, etc. The result of such curricula is usually a man with a 
broad amount of knowledge in most present day fields of application for OR. Unfor- 
tunately, however, OR is rarely discussed without mentioning the “team effort" or the 
"team approach." This implies a group of men working together on a specific problem. 
And, what of this group of men? Are they broadly trained, or are they specialists? 
Has their training been in breadth or in depth? 


PRACTICAL TRAINING 
An examination of efforts in OR, and also in QC, strongly indicates that the 
greatest success has been noted where the team effort was composed of capable and 
communicative scientists who were well trained in their own fields of specialization. 
A good OR team might consist of a physicist, a biologist, and a psychologist; or of a 


meteorologist and a mathematician. It is evident from the QC literature that the teem 


approach is gaining considerable favor. in! 6) it is suggested that a QC team might 
be composed of a shop supervisor, a product engineer, and a QC engineer. "Each mem- 


ber brings to the group his specialized knowledge." In another tiera'?? the QC team 
consisted of a research chemist, a technical service engineer, and a quality control 
engineer. Here, too, it was found that the interdisciplinary approach proved most 
successful. The inferences which may be drawn from the above situations are many. 
Most significant is the fact that many of the problems which were solved could never 
have been solved by a single man, either broadly trained or an outstanding specialist 
(7) 


in one field. The requirements seem to be 


1. To have specialists who have mastered their own fields well 
enough that they can work in new situations; 
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2. To have men with above average ingenuity, creativity, and initiative; 


and, 
3. To have a good interchange of ideas and commmication within the 
group. 


And how have these requirements been met in recent years? Not through most formal 
training programs! Instead, the unusually talented have been selected to become part 
of an OR or a Q@ team. The problem and the approach may differ but the basic ingre- 
dients are the same: the specialists working as a team. 


ELEMENTS IN TRAINING 
While the cries of agreement may not be loud, it seems that there are certain 
important elements necessary for the development of men who can serve as members of a 
team whether it be in OR or in QC. The elements are essential, in my opinion, to a 
good training program. These are: 


1. To obtain a sound training in a specific area of specialization. 

2. To maintain a close contact with reality so that obstacles and 
practical courses of action are clear. 

3. To develop enough flexibility so that stereotype approaches are 
not used for all problem situations. 

4. To acquire an ability to commmicate the specific subject area 
to others in an understandable way. 

5. To create a technique for conveying team and individual findings 
or conclusions in a clear and concise manner. 


These five points do not tell how to achieve the training requirements. They 
only point out what should be a goal in training for OR and QC team members. The 
specific training program must remain a function of the aims of the firm or the 
academic beliefs of the university. 


One final point, with respect to the importance of the individual trained in 
depth, is of significance. We must all remember that it is not enough to train others 
in the five areas indicated above. Those who make use of QC and OR teams must also be 
able to communicate their needs, identify and define their probléms, and to interpret 
the findings. It would be unfortunate indeed if the users of QC and OR teams were to 
fall into the same pattern of thinking many Americans have regarding foreign languages: 
let them learn our language! 
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OPERATIONS RESEARCH IN MANAGEMENT 


D. G. Malcolm, Vice President - Western Division 
Operations Research Incorporated, Santa Monica, Calif. 


The operations research approach has undergone a significant change during the 
past five years. For many years operations researchers have been occupied in des- 
cribing the various aspects of the organization under study. They have often appeared 
to be preoccupied with the improvement and sophistication of their understanding of the 
phenomenon of business and less concerned with systematic implementation or applica- 
tion of their research results. 


However, in the last 3-4 years many operations research activities have been 
directed to the study, improvement and “design" of systems of information--both 
planning and control in objective--by which management may discharge its responsibility. 
This paper is directed to a discussion of one such operations research study which has 
resulted in the design and wide use of a management information system used to plan 
and control large-scale R&D programs. 


The PERT information system, designed to provide progress information to the 
management team of the Navy Polaris program is a significant example of what modern 
industrial engineering, working in conjunction with operations research and computer 
specialists, can achieve. PERT, which stands for Program Evaluation and Review 


Technique, was created by a “system design" approach quite similar in its scope and g 
orderly development to the approach taken in the development of hardware items. It is = 
the purpose of this discussion to describe the basic PERT system, its current extensions & ; 
into the Cost and Reliability areas, and to touch briefly on some industrial applications Aes 


being made. 


PERT was designed to deal with the measurement and control of Time,ie., compli- 
ance to plans, scheduling, planning and prediction of progress. Other management 
research in this area, by the Navy, the Air Force and industry, is extending the PERT 
concept into measurement and prediction of Cost and Performance--where performance 
refers to the performance of the item under development. 


Admiral Raborn, Director of Special Projects Office which has managed the 
Polaris program, often tells his people that “If you can think out a plan, you can also 
write it down". This message supports the planners to encourage their technical staffs 
to reduce to writing the ideas they have in their minds. This is a necessary first step in ae 
“creating on schedule". Lt. General Schriever, Commander of the Air Force Research and eg 
Development Command which develops the many weapons systems of the Air Force, has 
stated that one of their most serious problems in space and missile development lies in 
the development of management methods themselves. These two men, encharged with a 
large portion of our creative R&D, have thus laid down a challenge to all of us who are 
working in the management systems area, to develop management systems which can 
keep up with the technology that we are rapidly acquiring in the development of missiles 
and space vehicles themselves. Both men endorse PERT, the subject at hand, as a tool 
necessary in the reduction of both time and cost in our defense effort. 


It has been the concept of “concurrency”, i.e., concurrent design and development 
in order to reduce development time, that has created the need for a new technique, pre- 
dictive in nature and able to cope with concurrent, interrelated activities thus created. 
The increased coordination and attendant information required are shown in Figure l, 
which illustrates the added number of interactions required to operate in a concurrent 
fashion. 


Admiral Raborn, in setting up PERT, has recognized the need for a significant F 
effort to develop the management tools that go along with the creative technical work 
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FIGURE 1 -- Shortening The Time for W. S. Development 


that is involved. Recognition of this problem and putting research time and effort 
against it is a rather unique contribution in and of itself, for it generally is quite diffi- 
cult to establish research projects in the operational management area. 


Since initial application in Polaris, the concept has spread through several Air 
Force, Navy and Army programs and is being used in private industry for the management 
of the new product activity, where savings on the order of 20% have been claimed. 


The Navy's program in PERT was part of a time-phased program much the same as 
we see in the development of a missile itself. We speak of generations of missiles, 
each one having greater range and other operational capabilities than the preceding 
generation. Likewise, in establishing management systems, the concept of going at it 
in a series of generations was conceived, and PERT represents the first such generation 
of a management information system, It tackles primarily the problem of Time compliance 
in a development program. Second and third generations of this management system are 
currently under development in the Navy: one in terms of Cost, dedicated to integrating 
cost or resource information with the time information; and another which is tackling the 
Performance, or reliability, measurement and control. In the latter area, it is hoped to 
provide a foreward look at what the reliability problems are early enough to make the 
most useful tradeoffs and reallocation of resources. 


A Description of the PERT Approach 


In the PERT approach, the development program is first portrayed graphically as a 
network of interrelated activities necessary to achieve prescribed milestones, or events. 
Figure 2 shows a typical network or flow plan for a small portion of the Polaris program. | 
Events are shown as squares in the diagram and activities as the connecting arrows. The 
“critical path" is the longest path through the given program. It is this part of the pro- 
gram that management is most vitally interested in determining, shortening and monitor- 
ing. The computer has been programmed to sort out from the many concurrent paths, 
which is the longest and next longest, etc. Figure 3 sums up some of the major 
definitions used in PERT 


FIGURE 2 --PERT System Flow Plan or Network 
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@ AN INSTANTANEOUS OCCURRENCE WHOSE ACCOMPLISHMENT 
MUST BE KNOWN AT AN UNAMBIGUOUS POINT IN TIME. 


ACTIVITY ACTIVITY = 


THE TIME-CONSUMING EFFORT,OR WORK NECESSARY TO PROCEED 
FROM ONE EVENT TO ANOTHER. 


@ A FLOW CHART MADE UP OF ONE OR MORE SERIES OF EVENTS 
JOINED BY ACTIVITY LINES TO DEPICT THEIR INTERDEPENDENCIES 


AND INTERRELATIONSHIPS. 
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FIGURE 3 -- Basic Definitions in PERT System 
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FIGURE 4 -- Relationship of Networking to Gantt Charting 


The relationship of the network approach developed and used in PERT and 
traditional Gantt charting is worth commenting on. Figure 4 illustrates this difference. 
In Gantt charting there is no dependency, or interconnection, between activities shown. 
Coordination functions and precedent relationships are not shown in Gantt charting; 
these are of major significance in large R&D programs where many activities must be | 
performed concurrently and be properly coordinated. Planning for these points and 
utilizing this plan in monitoring makes it more nearly possible to "create on schedule"-- 
a requirement in our current weapons system program. The use of the “network” is thus 
a significant innovation to the body of industrial engineering techniques. 


The next step in the PERT process is to obtain elapsed time estimates for each 
activity in the network from engineers responsible for their completion. Three estimates 
are obtained for each activity, representing the range of time which can be expected. 
These estimates--optimistic, most likely and pessimistic--are transformed into a 
probability statement indicating the chances of the activity taking different lengths of 
time to be achieved. The flow plan and time estimates are then fed into a computer 
which computes and sorts out the longest path from al! the possible paths to any event. 
All ocher’ paths to an event thus have "slack" in them and represent areas where resources 
may be reallocated. The path having zero slack allocated with it has beer termed the 
“critical path" (see Figure 2). Thus PERT is a "Management-by-Exception" tool providing 
the manager with information where slips are likely to occur and what their magnitude may 
be. It also indicates where slack exists in the program and is guide for reallocating 
some of the resources to reduce total program time. This identification of slack areas 
also forewarns the manager of where not to buy attractive time reduction opportunities 
that may be proferred. 


In Figure 5 typical data available to.a manager are shown. Use of the three 
estimates makes it possible to develep a probability index for meeting or beating the 
schedule. The magnitude of this number may be used as a guide for determining the 
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FIGURE 5 -- Outputs from Analysis 
relative seriousness of the potential schedule slip or for rescheduling. 


In operation, PERT is maintained and updated according to a regular plan. 
Figure 6 is such an operating description of PERT. Another feature of PERT is the possi- 
bility for “simulating” a change. The manager may introduce a synthetic time reduction 
and find out what would happen to the total program as a result. Often time-reducing 
changes do not buy equivalent time reduction in the program and may better be rejected. 
Many displays are possible from the data available in the PERT computer files. 
Figure 7 is illustrative of how critical paths at higher management levels in the program 
can be developed by aggregating the data. A variety of such reports are available. 


Figure 8 assesses the role of the computer in PERT. it is highly important for 
management to specify what it wants out of the computer, and not have the computer lab 
tell management what it wants. It has happened often in the use of the PERT technique 
that the computer lab can see many possibilities of analyses and outputs of interest to 
them, which end up complicating the situation and making it difficult for management to 
see the real simplicity of the technique. The “management system design function" is 
one the industrial engineer fills--a sort of buffer between management and the computer 
lab. He knows the needs of management and is able also to communicate effectively 
with the computer specialists. The role of the system designer is becoming better 
recognized as the requirement for improved management controls is made. 


Extensions of PERT 


Up to this point we have discussed PERT. Let us now discuss some of the 
problems involved in extending management systems into second and third generations. 
In the PERT approach, only time estimates are obtained. It is possible for each one of 
the time estimates to obtain a cost estimate. Generally speaking, it is recognized that 
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FIGURE 6 -- PERT System in Operation 


at any given level of performance, if a reduction in the time of an activity is desired, it 
will take a disproportionate amount of resources applied to the task to get it done quicker. 
Further, if greater performance is desired at a given cost, it will take longer to achieve. 
Put in another way, for any given level of time, if greater performance is desired -- 

i.e., a better missile -- it will cost more and/or require more time. Figure 9 illustrates 
this relationship. 


Early in the original PERT research, it was deemed impractical on two counts to 
cope with all three variables (Time, Cost and Performance) simultaneously in a computer 
model. First, related cost and time data on activities not experienced before are almost 
impossible to obtain with any degree of accuracy. It was almost impossible to get any- 
one to think significantly about such data. Further data on different degrees of the 
item's performance are even more difficult to obtain. Secondly, even if the data were 
obtainable, it would require a data processing load of about 20 times the data that the 
basic PERT system requires. Therefore, it was reasoned that if the total integrated 
Time, Cost and Performance approach was taken at the outset, the cost and data problem 
would defeat acceptance by the potential user of the system. It was decided to tackle 
the time variable first and then go after the cost variable after the information channels 
had been established. 
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SAMPLE PERT OUTLOOK FOR MAJOR FBM PROGRAM SUBSYSTEMS 
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FIGURE 7 -- Integrated Outlook 


A Principle in Management System Design 


One major principle in management system design that was followed in PERT and 
its extensions bears stating. This is the principle of taking a vertical slice of the 
program for the study effort and pilot installation. Figure 10 represents a technical or 
hardware breakdown of the Polaris System. The various subsystems are shown across 
the top and are divided into components, etc. It was decided in PERT that one had to go 
right down to where the work was done, i.e., take a vertical cut, rather than taking the 
' traditional horizontal cut in order to get immediate program results. In this way the 

ultimate system is completely consistent from top to bottom and one can have the 
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FIGURE 10 -- Vertical Slice of Development Program 
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confidence that data will be properly obtained and transmitted. After debugging this 
vertical slice, or pilot area, the system can be spread horizontally to other subsystems 
in a routine way. 


ie] n n 


There are a number of ways costs can be assigned to activities. One, a range 
of possible costs could be applied to each activity; or, a single cost for each one of 
the time estimates in PERT could be made. Further, there is a choice concerning whether 
to use a single cost, direct costs, or total cost. As shown in Figure ll, PERT applica- 
tions to date generally have applied direct man-hour costs, either in man-hours or 
dollars, to the activity, showing the department, code for the man, and in many cases 
identifying the individual man, where highly skilled individuals are a scarce commodity 
in great demand. Quite often in R&D, one particular designer is the only person who 
can do a certain kind of design job. It is desirable that he not be given too many tasks 
to perform concurrently. 


The vertical lines in Figure 11 illustrate the problem of making PERT cost data 
compatible with regular fiscal practices already in effect. It is seen that planned 
activities cut across the orderly monthly accounting periods shown. It is possible to 
convert from PERT activities to financial planning and accounting by knowing the rate of 
expenditure, but it is not possible to work in the other direction in the absence of a 
PERT diagram. This then is suggestive of the proper order of application. In short, 
PERT costs should be considered an input to current accounting systems. 


With data obtained as indicated, the following output reports are then possible 
in a company using the basic PERT cost approach: 


- Expected Manpower Requirements 
+ by skill, month and dept. 


Individual Man Loading 
by month 


+ Expected Project Direct Custs 
+ by skill, month and dept. 


+ Regular PERT Time Outputs 
+ slack areas 
+ critical paths 
expected calendar time, impact prediction 


Figures 12 and 13 show some of the types of data available from the cost system. 
It is noted that PERT Costs and Budgeted Costs may not always agree. Generally, PERT 
Costs will be lower and displaced in time due to the fact that all direct work may not be 
easily identified with networked activities. 


In summary, PERT costs are being used to determine manpower requirements by 
both skill, time period or month, and the department. In several applications, technical 
directors have found they have overlooked certain technical skills and this has set up 
the need for looking for additional people in that skill category. Individual man-loading 
by month, expected project direct costs by skill, month and department are available 
outputs. 


Cost outputs are being used in company planning for the following purposes: 
A, Determining and improving utilization. 
2. Balancing the work load. 


3. Evaluating Cost-Time tradeoffs. 
When the first PERT cost outputs are available and management has used them in 
improving utilization and balancing the work load, there will be other opportuni- 
ties to reallocate resources to activities or to apply new resources. The effect 

of these in regard to the over-all schedule, or time objective, may be easily 

evaluated by a simulated change, with the increment costs known. The effect- 
iveness can be measured in terms of the time reduction to be achieved. 
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FIGURE 12 -- PERT Costs vs. Estimated Costs 
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4. Determination of per cent of directed work. 
: In the course of planning any project there will be peaks and valleys in the 
ne requirements for the services of individual skills and individuals in particular. 
Where knowledge of this can be ascertained in advance, the time may be 
scheduled for other productive work that the company has available or desires 
to do--such as a directed research effort. Further, this per cent is a good 
project control device for management. 


5. Scheduling of manpower build-up. 
6. Identification and assignment of technical work. 


From the above it can be seen that the PERT network is capable of supplying basic 
information useful in planning, that this information is consistent with the definition 

of the planned and scheduled activities, and that it is possible to utilize this information 
in normal fiscal channels. It appears that this approach, which may be called “opera- 
tional accounting", in the R&D field, provides the necessary bridge or "common language” 
between planning and scheduling activities and fiscal accounting requirements. 


Performance 


Turning to the third component of the management problem, the "third generation" 
if you pléase, of the Polaris management system research activities--this has been 
titled Project PRISM, which stands for the Program Reliability Information System for 
Management. The objectives of the project are twofold: (1) To develop‘a capability to 
provide performance tradeoff information concerning the program approximately two years 
in the future, and (2) To develop a capability to quantitatively depict the expected 
reliability at the subsystem level. 


Z Two basic approaches in developing program reliability information are under 
consideration at this time by the project: 


l. D la e RMI method (Reliability Maturity 
Index). this method the as shown in Figure 14 is 
monitored to see that test specs, test procedures, acceptance procedures, etc., 
are written in accordance with the schedule set up. Each item of an assembly is 
rated in accordance with the factors shown in Figure 15. Check lists are built 
up for each of the rating factors shown, permitting a rating from zero to | to be 
assigned. The composite rating for each item represents an index of the 
reliability maturity. Management use of this information is directed toward low 
indices and zero compliance areas by appropriate sorting routines and report 
displays. 


2. System reliability prediction. The RMI method provides a measure of compliance 


and does not develop a numerical forecast of the eventual operational reliability 
ef the end item. Since one of our mest needed capabilities is the prediction of 
the eventual logistic requirement and operational up-time to be realized in 
practice, it is desirable to obtain a forecast number that can be used as a 
planning factor. The SP research program is developing a method known as RPM 
(Reliability Performance Measure) involving the following steps: 


a. 1 n f F s 
This model is used to estimate the per cent of successful launchings and 
utilizes data available from analyses of performance at each phase of the 
development cycle. 


b. Development of methods of analysis. 
Analysis methods are being developed which will predict reject rate and 
fleet failure rate, Different methods are required for preliminary design, 
design development, manufacturing logistics and operational phases of the 
‘development program. 


Cc. Vv. a m f fe) ment and subsystem estima 
into a total system estimate. 
The reliability estimates are made for various components, and subsystems 
are combined to make a total system reliability estimate (% successful launches). 
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FIGURE 14 -- Missile Development Cycle as Planned 
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FIGURE 15 -- Reliability Maturity Index (RMI) 


| 
| = 
| 


58 1961 ASQC CONVENTION TRANSACTIONS 


d. Development of an information system. 


A rapid means of collecting, analyzing and reporting information to 
technical managers is being developed. 


e. om son with reliability requirements. 
Using the predictions of item (d), it will be possible to compare the predicted 
reliability with the reliability requirements, or goals, set up in the original 
technical plan. Further, use of the computer model will permit the effect of 
changes in the failure rates of components and subsystems upon the total 
system reliability to be appraised. 


As these concepts are developed, it will be possible to provide management with a set 

of tools that will aid in enforcing the development plan, also a quantitative number pre- 
dicting the reliability of the item under design will be available at all stages in the 
development project. From this number it will be possible to detect weak spots in the 
program and to initiate specific remedies. Figure 16 illustrates the objective of the 
reliability information system to focus on the ultimate use at all times during development. 


In-Process 
Prediction 


Reliability 
Prediction 


FIGURE 16 -- The RPM Function 


Here again it should be noted that the management information system purports 
only to develop information and does not make decisions. In short, the concept is 
conceived to aid the decision maker and not to replace him. It should be further noted 
here that there are many pitfalls and problems in devising an appropriate reliability system. 


Industrial Applications of PERT Approach 


The PERT approach is being experimented with in a number of industrial areas. 
One that is under way in one of our iarge companies is attempting to utilize the PERT 
approach in the new product process. American industry is putting more and more money 
in research and has a major problem in determining whether it is getting a payoff on this 
investment. Further techniques for planning and controlling R&D functions are still 
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embryonic in nature. A way to predict and p'an for the timely and efficient exploitation 
of company research results is most needed. Figure 17 depicts the steps in developing 
a new product--a process that takes about 6 to 7 years for the average new product. 


____| THE NEW PRODUCT PROCESS | 
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FIGURE 17 


First, there is the task of searching out ideas and doing fundamental research. 
There are three areas shown in the Search and Research phase. The next phase is the 
broad area of development and test, or the development test phase, where selected 
ideas are brought up through reality of a pilot product. Coordination with the various 
other functional areas in the company is most important at this stage. The marketing, 
engineeriag and manufacturing areas all get involved because each has an eventual 
responsibility. Finally, if the test product is deemed feasible, it is moved into the 
commercial phase where it becomes a part of the product line. Out of 400 or 500 ideas, 
only one or two ever reach the commercial phase. 


The objective, therefore, in research and development is to attempt to get the 
new product process performed efficiently and in a minimum time. Yet, in many 
companies the mode of orgénization works against getting new products out ina 
reasonable time. 


Figure 18 represents such a typical industrial concern which is functionally 
organized--top management with sales, research engineering, distribution and produc- 
tion shown, each having a say about everything that goes on in current operations; 
research generally responsible for coming up with the new ideas and hopefully having 
them well enought brought out so they will move on into production and sales quickly. 


Across the bottom of Figure 18 this new product process is shown--the search, 
screen, specification development, testing and commercial phases. Superimposed 
over these phases is a PERT diagram in the aggregate that shows the searching for 
ideas, selecting of promising ones for further refining, specifying what is to go into 
development, performing development tests. Moving across to the right represents 
the passage of time, with the lines coming down from the top in spider-web fashion 
indicating that almost all of the functional areas get involved in each of the various 
phases. Generally the various functions are not involved in a coordinated way. 
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FIGURE 18 -- Project Planning in a Functional Organization 


The idea of creating a project management concept for the product early in the game and 
using PERT as the management tool to cut across the functional lines appears promising 
and follows logically after a project network is set up. This use of PERT will have an 
effect on the traditional functional organization. Some refer to this as turning the 
organization 90 degrees and focusing on the project objectives. A project manager is 
required who can reach out for the necessary staff and systems analysis he needs all 
throughout the program. In this way the whole organization can be coordinated and 
brought to bear on new product objectives. ; 


The companies that are involved in this confidently expressed the fact that they 
believe they can cut the cost of the development of new products by at least 20%, both 
in time and cost, due to better coordination between the functional areas of the company 
early in the game, on a planned basis, where everybody knows what their function is 
to be. 


Several other applications of the networking concept have been suggested-- 
long range planning, maintenance and construction activities, to name a few. 


In conclusion, the PERT approach offers opportunity for industrial engineers to 
develop better planning and control tools to aid in the management of the R&D process. 
Such predictive controls are vitally needed to unharness and realize the benefits of 
American R&D more efficiently as well as quickly. 
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PROCESS AND QUALITY CONTROL FOR MANAGEMENT 


Walter T, Hayter, Quality Control Metallurgist 
The Youngstown Sheet and Tube Co., East Chicago, Indiana 


At no time since the advent of the Industrial Revolution has there been so great a 
need for improved efficiency of production, By this we mean higher productivities, 
higher production, better yields, and better quality. 


This need has been brought about primarily by a rapidly changing economic climate. 


process of manufacture; foreign competition in the steel and other industries has made 
quality material available to the American market at selling prices often below that of 
our domestic manufacture, and dynamic foreign manufacture and sales have adversely ef- 
fected our export business. 


In addition, the high cost of new facilities and the unrealistic amortization rate 


Next we have the increasingly critical quality demands by our customers and the 
ffect of automation upon these demands, High speed facilities such as can lines mak- 
ing 600 or more cans per mimte, 

ver 
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minute and progressive die setups, demand more uniform quality than e before. 
of the results of this demand for more uniform and dependable piality has been no in 
the automobile industry. In 1961, automobile manufacturers found it necessary to in- 


crease the period of their product guarantee far beyond that of previous years, 


Last, but not least, we have increasing demands for more uniform quality on our 
own processing unit. As each of our finishing units is improved to produce at a higher 
speed - such as electrolytic tinning lines at 2500 feet per minute, continuous anneal- 
ing lines at 1200 feet per minute, cold reduction mills at 7000 feet per mimute, and 
hot strip mills at 2000 or more tons per turn - it becomes increasingly necessary to 
have a more uniform product of better quality entering each unit. 


The total of these increases in labor, transportation, facilities, and tax costs, 
coupled with low cost imports and improved product quality demands, forces upon us this 
need for efficiency and more efficiency. 


We, at The Youngstown Sheet and Tube Company improved the efficiency of our opera- 
ting units by application of a principle which we call “contimity of operation". 


This is a method which increases productivity, production and yields and enhances 
product quality available for shipment. In effect we have found it makes better use of 
equipment and man hours, Simply stated, this principle is that on any cont: 
producing unit, discontinuity of production during scheduled operating hours adversely 
affects costs, productivity and quality, Interrupted production results in less pro- 


ted in higher cost per unit of production. (Figure I). If such interruptions were to 
continue for a long period of time, more producing units conceivably would be required 
to produce a given amount of material. 


Discontinuity of production has a definite effect upon quality, This can be dem- 
onstrated by its effect upon electrolytic tinned strip for use in making tin cans, This 
type of unit produces tin plate in coil form, with coils welded end to end to form a 
continuous ribbon, Delays, or discontinuity of production for any reason results in 
overpickled plate, excessive tin coatings, burned, stained or damaged tin plate with 
the end result of more rejects and secondary plate. 


Discontinuity of production and resultant poor quality can result in customer dis- 
appointment and disrupted schedules in his plants. Material rejected for off quality 
due to production difficulties requires reordering and rerolling and causes delayed 
shipment. Broken promises are long remembered by a customer sorely in need of 
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¢ Labor gradually has been receiving a larger portion of the sales dollar; transportation 
costs have increased to a high level adding to the cost of all products used in the 
of these facilities pose new needs for greater efficiency, Actually the rise in the y 
stay even cost is increasing at the rate of billions of dollars per year. This is ? 
partly because capacity has expanded and there are more plants to be kept up. 
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Fig. 1 - Relationship between delays and productivity on a 54-inch hot strip mill. 
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material. This particularly is true in our present era of closely watched inventory 
control. Discontinuity of production, particularly in this day of large unit product, 
such as steel shivped in coils creates the possibility of customer complaints and 


Methods of achieving greater contimity of production were worked out in our : 
plants. These required careful analysis of production records - always searching for or 
basic causes of production discontinuity. In general, there were found to be 4 basic : 
causes of discontimity - mechanical, electrical, operating and outside as shown in 
Figure II, Most time was found to be lost due to the larger recurring types of delays 
such as bearings in the mechanical group, motor problems in the electrical group, roll 
or size changes in the operating group and power failure in the outside group. The 
balance of lost time in each group consists of many small recurring or non-recurring 
delays which can consume a considerable amount of down time. It was found that extrem 
care is necessary to see that all delays are categorized properly and that standard he 
terminology is used in recording delays. As in any work involving the use of statis- og 
tics, it is axiomatic that reliability of data is of prime importance. s 


Once our delay reports were set up, we set about to determine the individual ef- 
fects of given delays upon quality. This was done by careful studies of quality fac- 


nts and we, as quality control people, were able to concentrate our efforts to 
the direct quality factors and to report our findings so that corrective actions could x 
be taken. This required a period of transition during which reports were carefully and a, 
gradually changed. However, at all times, we emphasized that production discontimity 

Problems would have a pronounced effect upon our total quality. 


With our wide background of information on production variables, we used statisti- 


weekly, monthly and semi annual basis. These reports, in general, are divided into 
steel plant and flat roll quality reports. 


Information contained in the steel plant reports includes a relatively large group 
of unit and category data variables. 


1. Ingot to product yields by unit and class of product. 

2. Unit yields by product. 

3. Froductivities by class of product and total. 

4. A group of blast furnace figures such as 

a. Distribution of silicon values by blast furnace. 

b. Distribution of sulphur values by blast furnace. 

4 group of open hearth quality factors such as 

@. The percentage of leaky pours for each shop. 

b. Off heats by shop. 

c. Causes and percentages of furnace switches. 

da. Causes and percentages of diversions. 

e. Tap to tap and other furnace time factors. 

f. Deviations from prescribed metallurgical temperature specifications. 

g- Deviations from good mold preparation practice. 

h, Chemical variations. 

i. Tap to blooming mill delivery time variations. 

6. Steel cleanliness as indicated by microscopic examination, 

7. Steel problems in each of the various finished products. 

8. Merchant miil factors such as percentage inspected, diverted and rejected. 

9. Galvanizing coating wight test summaries. 

10, Causes of rejections ax defects of all finished products. 
Some of the items covered in our flat rolled reporvs are: 

1. Slab to product yield comparisons. 


w 


5 claims due to defective material which mst be shipped in order to retain the entity of nf 
the coil. 
& 
: tors found to be outstanding in our quality control records and yield figures. All . 
F were assessed carefully and fed back through proper reporting channels to minimise the iS 
; occurrence of defect producing delays. Bae 
Gradually the work of delay analysis and control was turned over to the Production 
4 cal methods to arrive at expected ranges of variation in percentages defective of the et 
; various products, These included norms of variation in solution values, sheet count, mal 
chemistries, gauges, etc., all of which are prime quality factors. This information a 
wes incorporated into our routine reports. ns 
: In order to keep management properly and promptly informed, the Quality Control ‘y 
s Department of The Youngstown Sheet and Tube Company issues routine reports on a daily, Be r. 
ia 
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Fig. 2 -General groupings into which all delays are divided. 
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2. Unit yield comparisons. 

3. A group of hot strip quality factors such as: 
a. Finishing and coiling temperature deviations from aims of specifications. 
b. Results of conveyor roll observations. 
c. Results of slab surface observatiom. 
d. Percentages finished product defective and causes of defects. 

4. Continuous pickler solution chemistries which affect special surface properties 
of tin plate. 

5. Cold reduced sheet mill quality factors such as: 
a. Distributions of finished gauge. 
b. lubricity of rolling solutions. 
c. Percentage of weld breakage. 
d, Distributions of Rockwell values. 
e. Fercentages defective and causes of defects. 

6. Batch Anne 
a. Distribution of individual charge temperature ranges. 
b. Temperature a distribution of individual furnaces. 

7. Continuous 

a. Variations in me gas flow percentages. 

b. Variations in Hp percentages. 

c. Variations in de 

da. Distributions of hoe 

Electrolytic Tinning: 

a. Tin coating weight control limits. 

b, Oil film percentages within range. 

c. Percentages of OK product off the shears and causes of held plate, 

d. Source analysis of waste-waste. 

e. Special surface property test walues such as Pickle Lag and Iron Solution 
values. 


These are only partial listings of quality and process factors listed in our sev- 
eral reports, However, it will serve to give an idea of the wide range of information 
disseminated routinely to all levels of management from foremen to vice president. 


The gathering and reporting of statistics certainly is not new to the steel indus- 
try or to any branch of man's economic life, As far back as 300 A.D., the Roman Emper- 
or Diocletian set his statisticians to work and they evolved a system of economic con- 
trols which were meant to be used uniformly throughout the Empire. Statistics concern- 
ing ingot tons produced, cost of production and amount of material shipped have long 
been published in the steel industry. However, the dissemination of a complexity of 
quality control information to all levels of production employees is a new idea. 


We are not sure how readable were the reports issued for Diocletian's edification - 
but we are sure that those intended for the use of all levels of management mst be in 
such a form as to attract attention to information pertinent to problems of manufacture 
and quality. A routine quality control repart which resembles a stock quotation list- 
ing with codes and excessive detail will not be used effectively. 


Our reports are presented in such a way that the busy man can identify danger 
areas immediately and just as readily detect areas of increased quality performance. 
This partially is achieved by the use of colored ditto and photographic papers using 
red for the areas which require attention and green for those which have improved. 
White papers identify those areas in which no changes have taken place and conditions 
refiect production of normal quality. 


The use of charts in our reports enables management to compare the period of the 
report with previous periods and note the effect of changes in equipment and other 
Process variables. In addition, each report has a cover letter which gives a one sen- 
tence summary of each major factor contained therein. Cover letters are divided into 
two sections indicating in one those factors which are in good shape and in the other 
those which are not so good. Our monthly reports have an additional cower page which 
contains a quality propaganda slogan such as "Quality Work Assures A Good Reputation" 
and a well designed cartoon which portrays the meaning of the slogan. 


In order to add to their effectiveness, quality control reports at The Youngstown 
Sheet and Tube Company are published to a schedule so that all information will be 
timely and fresh. We believe this is one of the reasons for acceptance of our reports 
by all echelons of management. They realize they are not looking at information so 
outdated as to be valueless in their current operations. 
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other effective methods are available for use in presenting the process and 


countenance no argument. The picture clearly indicates the factor producing poor qual- 
ity almost at the time it happens. Whenever possible, slides are shown in color - 
which is not as difficult as it sounds, There are simple vegetable dye coloring meth- 
ods which can be used on black and white negatives - and the results are attractive. 


The use of demonstration boards permanently installed in conference rooms is of 
great use to inform management. Large charts can be tacked to these boards and impor- 
tant quality and yield information kept before any group using the conference room. 
Quality is the business of every employee and no opportunity should be missed to pre- 
sent the quality story. 


In addition to presenting statistical information, there are other means of in- 
stilling the spirit of quality in all levels of employees, including management. At 
least once each month we publish a "Quality Gram" throughout the plant. These are one 
page issues containing an original message showing how cooperation, teamwork, effective 
quality. Our "Quality Grams" often contain a picture of the subject discussed or a 


To summarise our thinking on the matter of process and quality control, we are 
convinced that efficiency, productivity and quality improvements can be made 
the effective studies of and wide interplant publicity concerning production, effi- 


inherently wants to be proud of his work and that properly placed publicity will aid 
hix in doing so. 


Persistance in presentation and insistance upon conformity to standards, 
upon imagination as an aid and tool can provide the means of achieving the goal of ef- 
ficiency required for today's necessary economic manufacture. We believe with A. S. 
Osburn ® -——=- knowledge becomes more usable when imaginatively synthesized and dynamic- 
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quality stary to management. Use of the instant-pictures type of slides for quality and 
' production meetings often enables our Quality Control Department to demonstrate their 
, story with pictures taken in the mill only minutes before any meeting. Such slides 
ciency and adherence to standards set up through scientific studies. This best can be 
done by using accurate and imaginative techniques of gathering, interpreting and pre- 
senting data. Information gathered should be presented regularly in simple forn, 7 
: graphic means when possible, to everyone concerned from foremen to vice president. 
; of this is done to create an interest in doing a good job - remembering that every man 
ally extended.” | 
J. Wiley and Sons, Inc., 1947. 
SHEWART, WALTER A.: Economic Control of Quality in Manufac- 4 


GETTING STARTED IN STATISTICAL QUALITY CONTROL é‘ 


F, Alex, Gnaedinger 
Superintendent - Product Standards 
Canadian Celanese Limited 


You are at a "Basic Concept" session of the Fifteenth Annual Con- 
vention of the American Society for Quality Control. As the speaker 
commences, he is interrupted by a voice from a front seat - 

“My name is Harvey Blank and I have a question". - 

"Yes Sir" 


"I am the Supervisor of the Quality Control Laboratory of the Dandy 
Handy Co, Ltd. We make Phoiphoiners, you knew". 


"Yes Sir" 


"I received your mailed announcement. The theme ‘Greater Quantity - 
Better Quality - Lower Costs' certainly appealed to my boss. He's the 
Vice-President of Manufacturing. Naturally he's interested in reducing 
waste - making a better product and lowering his costs. We make 
Phoiphoiners, you know", 


"Yes Sir - But you had a question to ask". 


"Yes, I did. I think it's of interest to others like myself who 
are here to-day. What's the gimmick? I mean-how is it done? We al- 
ready have Quality Control established in our operations. We pride our- 
self on the quality of our product. We make Phoiphoiners, you know" - 


"Frankly Sir, Phoiphoiners mean nothing to me" - 


“Surely this isn't so. Phoiphoiners are these rounded dice for 
gamblers that would rather play marbles" - 


"I see - but to continue your question" 


"I'll agree that sometimes we run into trouble and our throw-outs 
are high, which result in higher costs. On occasion, we get the odd 
complaint. However, generally speaking, we do not do too badly. Never- 
theless my boss, and I, are curious. How can we do a better job?" 


“Well Sir, you have just completed the first step in finding out 
how to do a better job. That is, you've come to this Convention in 
order to seek help and information. Are you familiar with Statistical 
Quality Control?" 


"I've read some references to it in current trade periodicals. 
But statistics - statistical’ That is for the long hairs the Doctors of 
what-not, government bureaus and the like Isn't? We make Phoiphoiners, 
you know". 


"I know" 

“Besides the applications I've read about are in steel, aircraft, 
textile, chemical, business office applications - etc. Our problems 
are much different. We make Phoiphoiners. You know what Phoiphoiners 
are?" 

"I thought I did". 


“They are those small ladders, without steps, for washing the 
windows in the basement", 
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" I see" 
"I don't think that this statistical stuff is for us". 


"You know, Mr. Blank, it is surprising the variety of reaction 
that the words ‘statistics and 'statistical' produce in people, If you 
will pardon my observation, you appear to have reacted in typical 
fashion, Don't feel badly about it, however, Note that the Society, 
which is sponsoring this and other international meetings, has for 
reasons of its own thought it prudent to make no mention of the word 
statistical in its formal name. Rather, it is simply called the 
Society for Quality Control, Yet as you will see, it is from this 
word that the power and success of your undertakings will stem". 


“Are you saying, that my boss and I should become statisticians?" 


"Definitely not. If necessary, as you get into the thing, you may 
find it useful to employ someone with formal statistical training. But 
in the first instance, the most important thing is to have an apprecia- 
tion and understanding of the statistical philosophy or the statistical 
viewpoint. The beauty of it lies in its simplicity and universal 
application. Simply, it is a manner of thinking which does not require 
any prerequisite of mathematical or technical knowledge". 


“Then in getting started, in Statistical Quality Control, you con- 
sider that the most important thing is to recognize that a new viewpoint 
exists for handling problems". 


"Mr. Blank - you are so right. How deep you want to go into the 
subject will depend entirely upon your level of responsibility. Your 
boss will probably only want to become acquainted with the philosophical 
aspects. You will want to understand something of the statistical 
techniques and methodology". 


"O.K. so far, but how does this fit in to making Phoiphoiners more 
efficiently?" 


"That Sir, is precisely why I am here, If you would stop and 
reflect a moment, I am sure that you would agree that there is one by- 
product of any industry, business or processing operation that's common 
to all. This by-product is the numerical facts that are systematically 
collected, 

Facts in the form of figures appear in every phase of our operation, 
Production line personnel continually make checks on machine operating 
conditions, Temperatures, pressures, volumes, speeds, atmospheric con- 
ditions in the operating areas, etc. are measured and reported. 

Quality Control and Laboratory Technicians sample and test the incoming, 
in-process and out-going material, 

Industrial Engineers measure and express numericaliy man's productivity 
in units of out-put per unit of time. 

Accountant's figures measure and record the financial "Facts of life" of 
conducting the enterprise. 

You can readily see that the data sources seem to be different and limit- 
less. But it is more important to note that a common problem exists. It 
is this - how can we extract the proper meaning from all the figures that 
are collected, 

It is also important to reaiize that the problem does not recognize any 
industrial boundaries", 


“Well it certainly does become confusing at times trying to make 
sense out of all the figures that are collected and go over my desk in a 
day. These I guess are statistics". 


"They are, Mr. Blank, and sooner or later you will come to 
appreciate the fact that many of your problems do not lie in the tech- 
niques of manufacturing Phoiphoiners but are statistical ones arising out 
of the data you collect. It is a sad fact, but true, that most of us do 
not receive any formal introduction into the most efficient way to collect 
and utilize data. We have, perhaps, not distinguished the difference 
between the ability to account and do simple arithmetic, and the ability 
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to interpret the numbers and decide when it is definitely safe to say 
one thing or another. 

Fortunately this situation is easily remedied. You are particularly 
interested in quality control. There are many good books on the market 
to-day that cover the application of statistical methods to quality 
control problems. In addition, evening or day, elementary or advanced 
courses, sponsored by the Society or Universities are available in your 
community. Keep one thing in mind, though. Books and courses tend to 
emphasize the mechanics of the statistical method. Most beginners 
master these easily. In their eagerness to get started on the "better- 
ment" trail, they collect data, caiculate the required arithmetic means 
and measures of variation and set-up control limits as the manuals set 
forth. But in the process, they often fail to consider the underlying 
philosophy". 


"Well, what is this statistical quality control philosophy?" 


"You will agree, Mr. Blank, that the use of statistical methods 
implies collection of data. The philosophy of statistical quality 
control rests on the assumption that the collected data constitute a 
sample, and that on the basis of the sample, some useful decision can be 
made regarding the character of the data source together with a compari- 
son decision regarding preferable action on the source, 

This simple thought leads to a powerful method of investigation. For in 
the process, you will have to consider, what kind of data describes a 
situation which is in control, whether the data can be used to predict 
future performance, and how best to collect and use the data to illus- 
trate degree of control", 


"O.K. I buy my book, take my course, em steeped in the fundamentals 
then what?" 


"The next thing to do, Mr. Blank, is te find a starting place to 
put the new view-point to work. This will involve a general assessment 
ef your present manufacturing position. It is highly desirable that you 
examine the amount of money you are losing in waste, process reworking, 
or quality complaints. This study will pin-point the most profitable 
area in which to operate. The economic survey should also be supple- 
mented by first hand information from fellow production supervisors and 
your boss. However, the most important thing to remember is to pick 
only one specific problem and concentrate your resources upon it." 


“From your experience, where do you think the most profitable and 
effective operational area is?" 


“Undoubtedly the most spectacular advances are made when statistical 
thinking is used in conjunction with process control through product 
testing or visual inspection. It should be self-evident that the place 
to control product quality and minimize waste is when manufacturing is 
taking place. The hope of any control procedure is that trouble will be 
recognized promptly and then rectified, To do this, measurements of one 
kind or another are taken. The facts that are collected are used to make 
decisions. Decisions may have to be taken by the operator, the foreman 
or the department management. With your newly acquired statistical view- 
point you can help them make better decisions and thus contribute to more 
effective manufacturing practice. In addition, and this is important, 
with the use of charted data on the manufacturing line, trouble can be 
identified, anticipated and avoided", 


"Then how should I proceed?" 


“Having stated the problem you must ask yourself what is really 
taking place during processing. In order to find out, you will want to 
collect some vital statistics, usually resorting to sampling techniques. 
This in turn will lead to the consideration of some important questions, 
It is appropriate to mention some of them at this time. 

The approach, fundamental and logical as it may seem in retrospect, is 
sometimes missed by investigators not trained in the statistical 
Dhilosophy", 
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How to Measure? 

A leading question to be considered is, if and how can you measure 
the particular processing aspect that you are interested in. You must 
get things expressed in terms of numbers. This is simple when you are 
measuring things on numbered scales like weight, density, length, time, 
etc. or events that can be classified as good or bad, present or absent. 
Sometimes, however, abstract qualities must be measured and it will test 
your ingenuity to convert them into some type of numerical yard stick. 
As a laboratory Supervisor, Mr. Blank, you must consider whether your 
laboratory tests fulfill their intended end-use. Can you distinguish 
one situation from another with your test? You will want to know, not 
think you know, what the precision and accuracy of your tests are. 
Precision can be measured quantitatively. In many laboratories it is 
included as a part of the specification for the analytical procedure. 
Quality Control of laboratory techniques are just as important as 
process control. Many laboratories use a quality audit program with 
control charts in order to effect this. It is surprising when conduct- 
ing investigations to note the misconceptions that exist, in this area, 
by laboratory personnel. In some instances, production personnel are 
blamed for fighting losing battles whereas it is in the analytical 
laboratory that the fault lies". 


"I suppose that the same thiug is true for the visual inspection 
procedures on the production line", 


"Yes, Mr. Blank it is. Process problems often arise out of 
differences that exist in specification interpretation by visual 
inspectors, 

The point to be made now is, I think, that during the quality control 
investigation, the data producers must be studied, for the decisions 
to be made can be no better than the data that is collected", 


What to Measur3? 

There are some important things to remember when you are getting 
started in your process sampling investigation. All sampling enquiries 
are aimed at discovering something about a particular data source. Be 
sure you define the source that you are interested in, You may be in- 
terested in knowing what is happening on one spindle, one machine, all 
spindles, all machines, etc. Sample the right source and confine your 
conclusions to that source. It goes without saying that you should 
understand clearly what you are trying to find out about the data source. 
Remember that samples are usually of a limited size and so will contain 
limited information. By all means take into account any prior knowledge 
which can be derived from other sources, 

Mr, Blank, we've been talking a»out some of the necessary things to con- 
sider when getting started in statistical quality control. Perhaps, I 
can best sum it all up for you by means of an example. 


Case Histor 
The Carter Littie River Mills is a company that produces an inter- 
mediate product on 288 machines, Their particular problem was to 


control the number of times the machines stopped. It was generally 
recognized that niachines stopped, in a random fashion, for two reasons: 

1) Mechanical trouble (Mechanical stops) 

2) Raw Material failure (Material stops) 
Stoppages for nechanical reasons could be attributed to inadequate 
general maintenance or improper mechanical adjustment, 

Stoppages for reasons of raw material failure could be associated with 
quality problems arising out of preceding processes. 
The number of times a machine stopped affected three things: 

1) Machine Job Loads 

2) Quality of the finished product itself 

3) Waste arising out of rejects due to sub-standard quality. 

The Industrial Engineering Department established job loads based on the 
number of random stoppages that occurred during processing. It was an 
established fact that it took an operator one minute to rectify a situa- 
tion arising from a stoppage. The operators were scheduled to do 36 
minutes of rectification work in every hour, Thus, if the random 
stoppage rate was one per machine hour, an operator was scheduled to 
operate 36 machines. 
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Quality of the finished product was judged through visual inspection. 
Each time a machine stopped, a defect occurred in the product. 
Tolerances for the number of defects per production unit had been 
established. Any production unit containing excessive defects was 
rejected and became waste. 


It is evident, then, that high machine stoppage rates were of great 
concern to the plant management. When the rate was high, job loads were 
low and labor operating costs increased from the fact that additional 
machine operators had to be employed to run the full machine complement. 
In addition, finished product quality deteriorated due to the high 
incidence of defects in acceptable product. High costs resulting out of 
excessive waste also accompanied this situation. 

Plant Management attempted to control their operation through the use of 
"stoppage checking" techniques. "“Stoppage Checkers" were employed who 
went out into the plant and made two-hour checks in the operating area. 
Their assignment was to observe the number of machines tended by one 
operator. They noted the number of times each machine stopped during 

the check interval. They also attempted to assign the reason for the 
stoppage, that is, whether it was for Mechanical or Material reasons. 
After each check was made, the check sheet was handed to the department 
foreman who was responsible for seeing that the report was followed up. 
If the mechanical reasons seemed to be excessive, the mechanic in charge 
of the offending machinery was notified. 

This was the well-established control technique. Generally speaking it 
was most effective. However, the day arrived, when the Higher Management 
passed the word along to the Plant that under to-day's business con- 
ditions, operating costs were considered to be too high. Labor costs 
were pricing them out of the market. Complaints of quality, arising out 
of competitor comparisons, were beginning to come in. Something had to 
be done. 

The machine stoppage problem was referred to a Quality Control Technician 
who had statistical Quality Control Training. His first task was to take 
over direction of the “stoppage checkers." In doing so, he noted: 

(1) some checkers could not properly identify the stoppage 

reason, This was rectified through training. 

(2) although the standard check time was supposed to be 
two hours, the check times ran from 1 1/2 - 2 1/2 
hours. The time was standardized to two hours, 
(3) times of checking were not done at random intervals. 

This was rectified through the use of a set of 

random numbers, 
The sampling program was started in such a way that the entire machine 
complement was covered every week. To facilitate analysis of the data, 
a Control Chart was started for each machine on which was plotted the 
result of each two-hour observation. 
The total stops were charted on the top part of the chart and underneath 
it, a breakdown of the reasons, mechanical or material, were given, 
The usual method of handling the check sheets, with the production 
personnel on completion of an observation were followed. That is, after 
every check, the Department Foreman and/or the responsible mechanics 
received a copy of the results to follow up. 
It soon became apparent to the Quality Control Technician that the 
greatest reason for stoppages arose out of mechanical difficulties. 
Why was this? 
In discussing the question with the Department Foreman and the mechanics, 
it was established that they had a rule-of-thumb that they did not 
service a machine unless the stoppage check on a machine indicated three 
or more total stops. They had arrived at this arbitrary rule out of 
experience. They had gone out to check the mechanical condition of the 
machines on some occasions when 1 or 2 stoppages showed on the observa- 
tion sheet, only to find that in some cases, nothing was wrong. Any 
sheet with 3 or more stoppages appeared to be a safe bet. 
This was the clue that the Technician was looking for and he took 
immediate steps to rectify the situation. The Control Charts for each 
machine were taken down to the operating area and discussed with the 
personnel there. It was explained to them, that for every stoppage rate 
level, there is a pattern of varying sample results. A diagram, similar 
to Figure I, was prepared. From it they could see the difficulties that 
arise, in attempting to make a decision for action, based on a single 
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observation, 

It was obvious that a 2.0 average stoppage rate could easily be identi- 
fied, The problem of differentiating between a 1.0 and 0.5 average 
stoppage condition was not so obvious. Each condition could produce 

a number of single observations with 0, 1, or 2 stoppages. They soon 
appreciated the fact, that single sheets with these observations on them 
did not necessarily indicate that nothing was amiss, or that improvement 
was not impossible. It was pointed out that a number of results, or the 
pattern of variation of the sample results, had to be considered. This 
pattern could be predicted by reference to some useful tabulations, 
Control limits could be set which would define the pattern to be 
expected. When the predication was not met, something should be done 
about it. Subsequently summary charts were prepared by machines. 

Action was taken on the machines with high stoppage rate levels. A 
mechanic's training program to improve mechanical maintenance work was 
instituted. From this and the use of charted data, improvements 
immediately took place, 

Stoppage rates dropped to 0.5 stops per machine hour on most machines. 
It was possible to reassign job loads and reduce the labor costs by 
approximately 75%. Quality improved in finished acceptable end product 
and waste was reduced considerably. 


"Mr. Blank, such an example is not unusual in Statistical Quality 
Control Work,The superior results were not attained through complicated 
statistical methodology - but rather through a new view point - a manner 
of thinking, which does not differentiate between whether or not you are 
making textiles, aircraft, electronic components - or even phoiphoiners. 
And this, Mr. Blank, is just about where you and I came in". 


Author's Note 


The reader may wish to pursue the study of statistical philosophy. 
The following references are given, from which ideas have been borrowed 
freely. 


(1) Dr. E. G. Olds, “Some of the Challenging Questions in 
Statistical Control". Industrial Quality Control magazine. 
Vo. XII, No. 1. July 1955. 


(2) M. J. Moroney, "Facts from Figures", published as one of 
the Pelican Book series, A236, 
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BENEFITS OF QUALITY CONTROL INPUTS IN PREPARATION OF SPECIFICATIONS 


John E. Trader, Chief, Product Quality Assurance 
CONVAIR( ASTRONAUTICS) DIVISION 
GENERAL DYNAMICS CORPORATION 
San Diego, California 


Webster's dictionary defines the word "Specification" as, "A statement containing 


a minute description or enumeration of particulars or details of construction not show 


in an architect's drawings." 


The benefits or goals that might be cited as derived fram Quality Control input in 
the preparation of any specification are numerous. To highlight but a few: cost, 
schedules, design and re-design, contracts, employment, facilities, etc. However, 
the main benefit where long range quality planning pays off, is the overall prevention 
of quality problems. 


A coordinated team effort, consisting of Quality Control, Memfacturing Control 
and Product Design creates the perfect harmony prior to the initiation of any 
analysis of the product design for conditions that might otherwise create inherent 
quality deficiencies. Quality Control further assists in identifying the environ- 
ments and end-item conditions against which the product will be evaluated. This 

is of course reflects similar designs, failure histories, field and in-service 
feedback which is comsistently researched and evaluated by Quality Control. This 
information is then directed toward new specifications or modification of current 
documents. 


Quality Control can coordinate with design groups to determine campleteness 
of the test plan, including the tolerance camditions. By reviewing proposed engineer- 
ing evaluation tests, Quality Control can estimate fram adequate detail data the 
probability of the product to perform within established tolerances. Here Quality 
Control pre-preparation also highlights the probabilities of tightening or relaxing 
tolerance parameters for practical manufacturing purposes and environmental or 
acceptance level testing conditions which mainly reflect savings plus reliability 
to both the manufacturer and the customer. 


Quality Control can coordinate with design engineers to accamplish a practical 
application of the classification of characteristics as required, such as major and 
minor. 


Still another area for Quality Control in-put regarding specifications is the 
inclusion of data requirements for ensuring vendor participation, compatibility and 
capability of measuring systems, accuracy and characteristic variability. 


An additional element is the verification of the completeness of specifications 
which often times leave much to be desired. For example, one of the undesirable 
or vague elements to consider would be the possibility of misinterpretation. Here 
again benefits are very high due to the prior practice of Quality Control reviewing 
drawings and assisting in specification writing. Consider for a moment misinterpre- 
tation affects on the custamer, vendor, machinist, foreman, or inspection personnel. 
Just a little foresight, in-put and review will avoid this type of costly pitfalls. 


The responsibility for planning and outlining detailed test procedures and test 
equipment is yet another in-put achieved by the Quality Control Department. 


Quality manufacturing measursments and controls must be closely integrated and 
documented for maximum econamy and quality prior to manufacturing process. 


Other details covered by advanced Quality Control in-put are jig and fixture 
activities and controls on the calibration and measuring of test equipment and 
techniques to be utilized, certification of adequate personnel, production employee 
training and the initiation of new manufacturing processes and the like. 
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In the case of Astronautics, a Quality Assurance Group, within Quality Control 
is responsible for directing the following activities; 


1) Control Standards; Rules established by industry and/or Convair 
es for factors of quality, weight, value and quality 
standards are used to assure tint an acceptable quality level is maintained 
throughout inspection and testing procedures. Standards are prepared 
3 and issued in document form, usually as a specific standard applying to 
all items in general within the subject. 


2) ity Control ts; Accept proced prepared by Quality 
° ement specifications. Requirements are issued in documents, 
equipment operation procedures, or similar media. These documents are 
issued as part of the basic specificatiam and in case of sub-contractors it 
becames part of the procurement package. 


3) ty Control Document: This serialized document which is referenced 
immediately upon release, taking precedence 
over all other Quality Control procedures and standards pertaining to the 
items to which it refers. 


4) C Testing: Tests performed om individual items and individual lots 
su or acceptance under the contract. The tests verify those 
‘ conditions which show conformance with specified requirements of Convair, 
vendor and/or custamer. 


5) Production Evaluation Testing: Are tests which use quality “sampling” 
ques for tes pr tion units to verify and maintain proper 
production techniques, skills and tools. ‘They are essentially periodic 
equivalents of a “preproduction” (qualification) test. They expose the 
unit under test to all or a portion of the critical operating conditims 
and environments for which it has been designed. 


The point of interest here is the Quality Control document, wherein specifi- 
cations and measurement of quality or reliability is significant mmly in that 
means are available for its cmtrol. Such control mst take place throughout 
the entire cycle of development of the item fram conception om. This is recog- 
nized now by the number of specifications which call for reliability efforts to 
begin at the time of camcept development. 


It is obvious that these quality documents result in additiomal overhead and 
labor than would normally be applied without them. As ea comsequence, there is a 
tendency to comclude that this effort is too costly. Investigations into this 
phase however, indicate that this is not quite true. While direct assigned time 
and labor increases, the trend now clearly shows that without the coordination 
of Quality Control's prior inputs to specification writings, inherent 
and costly reworks multiply. 


It is quite obvious to us then that the sophisticated state of the art and 
the current electronic age is advancing rapidly. Quality Control has achieved 
and will continue to provide significant contributions and accamplishments to 
industry it services and to the public that pramotes competitive business activities. 


No longer is Quality Comtrol a “"behind-the-scene" operation, and better still, 
it has ceased to be a paralleling activity; for today we find it to be a forerunner. 
It is active and cmmtributes to project conceptions throughout development and 
manufacturing and always current with product life expectancies. 
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WHY QUALITY REQUIREMENTS MUST BE UPGRADED 


James L. Studdard, Special Assistant, Quality Assurance Division 
Army Ballistic Missile Agency, Redstone Arsenal, Alabama 


A. INTRODUCTION 


The psychologist tells us that we need to know something of the man if we hope to 
understand his thinking - his philosophy. There can be no doubt that 24 years of 
Federal Government service, with the last 9 dedicated to quality endeavors of U. S. 
Army Ordnance rocket and missile programs, will have its impact on a person's out- 
look. This is the background ty which you should evaluate this paper. 


To assure a basis of mutual understanding, it is often necessary to establish 
certain premise, As my thoughts are presented, please keep the following in mind: 


1. Whenever "quality requirements" are mentioned, think in terms of all those 
things that go towards providing a complete quality program. Quality requirements 
are not just those things normally found in Section 4 of a specification, 


2. Whenever "programs" or "missile programs" are mentioned, they will be in 
terms of complete, integrated, functional systems; consisting of both the missile and 
its associated ground equipment. 


3. Whenever, "quality" or "quality assurance" is used, it will be in terms of 
the Army Ballistic Missile Agency's (ABMA) “total quality" concept; not just the 
industrial production phase, 


It is recognized that many of yo will immediately associate some of my thoughts 
with “reliability.” Please bear in mind, my considerations are concerned with function 
- not organization, To us reliability is a phase - a most important phase - of a 
total quality program. 


B, QUALITY REQUIREMENTS 


ABMA's program of "total quality assurance" is certainly not new in concept, nor 
is it peculiar to Army Ordnance. Dr. A. V. Fiegenbaum has talked and written most 
persuasively about “total quality control" for several years. I trust you will find 
ABMA's approach and application of “total quality" to be of significance, and of mre 
than passing interest, 


With this, and prior premise, out of the way, let us consider some of the major 
“quality requirements" that make up the "total quality" package, and perhaps identify 
them not only as to "why" of upgrading, but also "what" and a bit of “how." 


Design & Development 


Just as “reliability” has its birth during development, so does "quality," and so 

must "quality requirements" - - a vigorous program of design - test - mocify - retest - 
t. A "total" design program must cover more than just the mechanical and/or 

electronic type considerations of the component parts that go together to make a 
functioning system, Out design people must also have knowledge of the total picture - 
the complete operation. There must be an acute awareness that eventually their “brain 
child" will be exposed to the many vagaries of the logistical supply line, and to the 
rugged conditions of the battlefield, The designer must keep in mind at all times 
that this wonderful missile system cannot be always tested and fired by engineers 
under laboratory conditions; but instead by soldiers under the stress and strain of 
battle conditions. If the designer can keep these things in mind, along with technical 
performance requirements, we will have taken a major step towards our quality goals. 


This is just one aspect of what we need to do in this area, Of equal, or even 
greater importance, are the quality standards and programs being developed by the 
quality engineer (contractor or Government), Now is the time to lay the foundation 
for the advancement of new quality knowledge - new methodoligies - new policies. 


ASQC LCS Code 342:70:991 


+ 
. 
4 
| 
fy 
77 
= 


“4 


78 1961 ASOQC CONVENTION TRANSACTIONS 


Production Control & Test 


Of all the many things we generally accept as being good quality practices, an 
area of particular significance is the quantity and quality of testing. The signifi- 
cance of the test equipment associated with our programs is of a magnitude barely 
equalled by the missile itself, and in many instances test equipment complexity exceeds 
that of the missile system. This being the case, isn't it rather incongruous to place 
so much emphasis on testing under Section 4 of the specification, and little, if any, 
thought to the test equipment, test personnel and test techniques? 


If I may digress a bit, and go back to design and development; here too is 
another significant area requiring action during development. The design and testing, 
and test equipment therefor, of complex missile systems is closely interlocked and 
interrelated - we can't have one without the other. In those areas where the hardware 
is to be functional - not static - this relationship is of paramount importance. The 
design engineer creates his conception of what is needed to perform a function. At 
the same time, he creates a bread~boar' testing device, Only with the marriage of 
these two is he able to determine whether or not his objective has been achieved. If 
the answer is positive - he is ready to mve forward. At that point in time (at the 
very latest) the quality engineer needs to be aboard for several reasons: 


1. To assist the design engineer in reducing to writing and/or drawing all of 
the test requirements that have been generated, (Those that are related to test 
equipment and test procedures.) 


2. To start the development of that portion of the procurement package (Section 
4 of Specifications) that will be needed for industrial procurement, To make certain 
the designer's intent is properly reflected in describing the item - its functional 
requirements and the acuteness of its characteristics ~ the test equipment and test 
procedures to be used to determine that "intent" has been met. 


3. The initiation of work for an orderly transition of test equipment from 
"bread-board" model to Industrial production; standardization - calibration - com- 
patibility - human engineering - automation. 


Throughout the program the test equipment must keep pace with missile hardware, The 
introduction of the quality engineer at a very early stage of development will provide 
assistance to the designer, and at the same time will assure orderly transition from 
development to full scale industrial production, 


The development of realistic requirements for test equipment is a major area 
where timeleness of action and upgrading are long overdue, Adding realism to this 
portion of the quality requirement, based on the many essentials learned through pro- 
duction, test, inspection and evaluation, is one of the major contributions Industry 
could make at this time. 


While we are in this area, I would like to mention briefly another activity where 
help is needed, Unfortunately some of our specifications came into being before we 
realized fully the high quality and reliability requirements needed for our missile 
and space programs, Valuable information is being gained daily through failure 
analysis. More and more is being learned about functional tolerances and the signifi- 
cance of various characteristics. All of these findings need to be reduced to complete 
and definitive specifications, Here again is an opportunity for Industry to perform a 
really vital service, We earnestly solicit your help towards keeping our specificat- 
ions at a level consistent with the needs of the "state of the art." 


Surveillance 


There are some who say quality functions, and associated quality requirements, end 
when the product goes out the door, Not so in the missile game. If we are to know 
where we stand, and if we hope to ever elevate the quality, or if you prefer relia- 
bility, plateau, we must place more emphasis on surveillance and maintenance of our 
equipment. Surveillance to see just what has happened during transportation, storage, 
operational check-out, modification and maintenance, Our basic responsibility as 
quality managers - quality engineers - first and foremost is to our Armed Forces, Such 
responsibility must provide demonstrable assurance to them of the quality, reliability, 
usability, and maintainability of our weapons systems, Demonstrable evidence under the 
most adverse conditions - not the conditions of laboratory, factory or proving ground - 
but under the conditions of combat, with the battlefield being world-wide. 
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Management 


By now you may be thinking - "these things are true, and most certainly they are 
important, but you are supposed to be discussing quality requirements, and requirements 
are things expected of someone." We readily recognize those things that can be re- 
duced to an exact science, but the thing that is really giving us trouble is the “art" 
of management. How does one go about the management of such ar activity? Can we 
reduce such quality requirements to writing, and make them a part of a specification or 
standard, and ultimately a contractual requirement? The more I see of our day-to-day 
operations wherein we are using, or attempting to use, specifications of suspect 
quality (adequacy), the more I am convinced that regardless of what might be written 
in a specification, or how well couched it might be in legal terms, unless it repre- 
sents ultimately a mutual agreement between the supplier and the buyer, we might as 
well forego the whole thing. I am convinced the things I have discussed can be reduced 
to meaningful writing, and they can be made contractual, The Department of Defense 
specification MIL-Q-9858 is an excellent base for such a document, For various reasons 
MIL-Q-9858 had to be a very general quality requirement. However, I am quite sure this 
Society (ASQC) will have the Department's blessing, if we can come up with a supplement 
that will provide for better quality and reliability. This need should be a challenge 
to us - ASQC - Industry - Government. Will we meet the challenge? 


By now some of you are probably saying - "this fellow is a dreamer. What he says 
is fine, but how in the world would such a program be administered?" We will do it by 
recognizing the program for what it is - A Top Quality Management Job. Place the 
quality function in its proper perspective, be it in Industry or Government, Recognize 
the problem as one requiring peculiar talents; engineers and specialist who devote 
their entire effort to quality activities. 


So, what may we conclude - where do we find ourselves? Certainly many of the 
things I have said are restatements of basic essentials long advocated by "quality 
minded" people. Many of them cannot be legislated into being. As a minimum we must 
introduce new concepts, and continue insisting that quality requirements be upgraded 
to reflect current needs, Anything less would be a disservice to our national well- 
being. 


C. SUMMARY 

If a summary is needed, remind yourself to: 

a. Always think of quality requirements from the "total quality" concept. Con- 
sider all of the things that must go into the program to give the troops a weapons 
system on which they can rely. 

b. Maintain specifications at the highest possible level. 

c. Insist on the proper status for the quality effort. 


d. Require fully qualified and wholly dedicated people to operate the quality 
Program. 


e. The management of quality programs is not a science; rather it is an art. 
Perhaps poorly defined at times, but nevertheless a useful art - provided we really 
work at it. 
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SOME RECENT DEVELOPMENTS IN PROCESS CONTROL 


Richard A. Freund 
Management Systems Development Department 
Eastman Kodak Company, Kodak Park Works 


During the early stages in the growth of statistical quality control, primary } 
emphasis was placed on techniques that assumed a nearly static set of production condi- if 
tions. It was assumed that the process was running steadily at some fixed level unless 1 
an assignable cause entered the production system to suddenly shift the process. The 
Shewhart control chart (13) is probably the best known technique developed on this 
basis. 


In recent years, attentic as been shifting to the problem of process control 
under a set of dynamic, rather iran static, conditions. Here it is assumed that some 
assignable causes are not merely unique occurrences that must be prevented, but are 
basic factors in their own right. For example, level shifts are not unexpected when 
different batches of raw material are fed into the process. Barnard, in his paper 
"Control Charts and Stochastic Processes" (2), suggests that the distribution of these 
"jumps" in level can be determined and corisidered in establishing a good control 
system. Where the effect of variations of this type must be minimized, Page has de- 
vised a new and extremely efficient technique known as the Cumulative Sum chart (7, 
11). This technique has now been developed to a stage where it can be used with ease 
by both line and staff control personnel. Truax (14, 15) has illustrated this in 
reports of his experience with Cumulative Sum charts at the Atlas Powder Company. 


In one sense, this approach can be viewed as a combination of control based on the 
most recent point plotted, as in the Shewhart chart, with an analysis of the trend of . 
previous observations. This is not the complete story, but let us first examine this - 
phase. There is a very strong similarity to sequential analysis. The deviation of : 


each current observation from the nominal value is added to the sum of the previous Bs" 
4 deviations; these sums are plotted and compared with control limits. The limits are ‘a 
treated in a novel and efficient manner suggested by Barnard: a V-shaped mask, point- 


ing forward with its axis parallel to the time scale, is placed a distance, d, ahead of es 
the latest point, and as long as all previous points remain visible within the mask, i. 
the process is considered in control. Two parameters, the angle of the V and the lead ¥ 
distance, d, must be specified. These parameters are functions of the « and PB risks of 
Type I and Type II errors, although a relatively new concept is used that replaces a 
direct consideration of these risks. This is the Average Run Length or A.R.L. (1, 5, 
7). The A.R.L. is a measure of how long the process will run at some level before 
process adjustment is called for. 


A good method for comparing various control systems is through the use of curves 
of Average Run Length functions, in the same way in which Operating Characteristic 
@.C) curves are compared. If the Shewhart chart is compared to a Cumulative Sum chart 
having an equivalent % ricx, the Cumulative Sum chart will show a superior performance 
in the sense of shorter average run lengths for small and moderate shifts in the 
process level. That is, the Cumulative Sum chart will call for process adjustment much 
earlier than the Shewhart chart for process shifts less than about three sigma units 
from the nominal value. Sudden large shifts can be detected more quickly by the 
Shewhart chart, but the difference is small here since both systems will respond rapid- 
ly- The advantages of the Cumulative Sum chart in respect to smaller average run 
lengths will be somewhat reduced when various schemes for making use of run theory 
supplement the Shewhart chart, but few such combinations have A.R.L. curves that close- 
ly match those for the Cumulative Sum chart. Proponents of the Cumulative Sum system 
point out that the Cumulative Sum procedure has a single operating criterion and thus 
is easier to handle since it avoids the complexity of supplementary rules for runs. 


Perhaps the most efficient use of run data is that of the "Control Charts With 
Warning Limits" previously proposed by Page in 1955 (9). The three sigma (or k sigma) 
Shewhart control limits are called "action" lines. Any single average plotting outside 
the action line calls, as the name implies, for immediate corrective action. In addi- 
tion, "warning" lines are drawn at some k' sigma level, (k'< k), for example, at 2 a 
sigma. If an average plots between the "warning" and "action" limits, the process is Yo 
suspect, but further information must be obtained before action is required. Two ' 
decision rules that could be used would be (1) that if n consecutive points plot 
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between the two lines, action will be required just as for the single point outside the 
action line, and (2) that if n out of N points are in this zone, action will be called 
for. A reasonable analogy is to consider the Shewhart chart as being similar to a 
single sampling plan, while the control chart with warning limits can be likened to a 
double or multiple sampling scheme. This latter scheme has the very appealing virtue 
of calling for immediate action on clear-cut process shifts, but giving careful consid- 
eration to borderline cases. The Cumulative Sum chart can then be compared to 
sequential sampling which is the most efficient method in the sense of getting complete 

‘A information from a small sample. In sampling inspection, the choice of single, double, 

y multiple, or sequential plans depends on a balance of the cost savings due to sampling 
efficiency and the expenses involved in administering a testing program in which the 

F size of the sample is a variable. The same considerations should hold when selecting a 
control system, but the administrative problem here is that of balancing the costs and 
risks involved in the length of time that the process can continue undetected at the 
wrong level, rather than how large a sample is needed at any given time. 


An additional virtue, and one of the most important preperties of the Cumulative 
Sum chart system, is its ease and efficiency in estimating the new process level should 
a shift occur (2, 11). The cumulative graph exhibits many advantages for visual inter- 
pretation of the data, both on a current basis and for post-mortem interpretations. 


Another system of accumulating deta to benefit from trend analysis has also been 
\ independently proposed recently. Roberts (12) has described a control chart test based 
on Geometric Moving Averages. Morris and Howe (6) have presented a system based on the 
same principles, using a different method of computation. Basically, both of these 
A papers suggest that the latest observation is the most important single bit of informa- 


- tion. Previous data should be averaged with it to take advantage of the continuity of 
*” the data and the process, but as the data gets older it should be weighted far less 
F than the current point. Unfortunately, not much actual experience with these systems 


has been reported so that a full comparison with the Cumulative Sum chart is not 
possible. As in the Cumulative Sum system, the Brisk is not defined by name, but the 
selection of the weighting constant effectively determines it. Again, the Average Run 
Length curves are better for these systems than for the straight Shewhart chart, when 
small and moderate level shifts occur. It is essential to point out that both Page and 
Roberts clearly state that these tools should be used only when greater sensitivity in 
detecting these small and moderate process fluctuations is required, and can be harmful 
in the sense of over-control should this not be the case. 


It is also interesting that to date there has been a strong tendency to select the 
limits for Cumulative Sum and Geometric Moving Average charts on the basis of experi- 

4 ence, rather than mathematical computation. Thus, the angle and lead distance of the 
V-mask for the Cumulative Sum chart is often obtained by plotting past data in cumula- 
tive form and shaping the V so that it catches known out-of-control situations, and 
does not reject periods of good production. The weighting constant for Geometric 

; Moving Average charts can be similarly obtained by revi past data. Of course, 

2 limits for each method can also be computed. See (5, 6, 125 for selected cases. 


As was pointed out above, the Cumulative Sum and Geometric Moving Average charts 
have been developed to meet the need for greater sensitivity that is found in 
situations. The auther in his papers on Acceptance Control charts (3, 4) has tackled 
the problem of control where greater sensitivity is neither required nor desirable 
because of the problems caused by over-control. Here again, recognition is given to 
the fact that there are certain assignable causes that act to keep the process away 
| from the nominal value, but against which protection is not feasible nor desirable. 

‘ Nor is the exact distribution of the "jumps" in process level of concern. 


In essence, the approach taken is to combine some of the aspects of acceptance 
sampling by variables with the control procedures. It is not unusual to find people 
using a control chart to decide whether to accept (continue to run) or reject (shut 
down or adjust) the process. Yet the Shewhart chart was not designed to be used in 
exactly this sense and no consideration is taken of the B risk of failing to detect a 
level change of some given magnitude. Under the Acceptance Control system, the amount 

® of process shift we wish to accept without adjusting the process is defined along with 
an @% risk of a Type I error. This, in effect, allows the line for the nominal process 
level to be expanded to a zone of acceptable processes. In addition, the amount of 
shift we wish to be sure of detecting with only a B risk of a Type II error is also 
defined. From this, the required sample size and control limit can easily be computed 

e assuming that we have enough information to get a reasonable estimate of sigma, the 

> within rational subgroup standard deviation. Sufficient experience has been accumu~ 

lated to show that this type of chart is extremely easy to use. Of even greater 
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importance, because thought has been applied in the selection of acceptable and rejec- 
table process levels, it is being used more rationally than had previous systems. It 
has proven particularly useful for batch type operations and has reduced over-control 
problems in both batch and continuous process applications. One virtue is great flexi- 
bility; the system can easily be supplemented or modified for special situations. For 
example, a Cumulative Sum chart, centered at the Acceptable Process Level or some level 
between the Acceptable Process Level and the Rejectable Process Level, could be added 
whenever n out of N points exceed the Acceptable Process Level, indicating that the 
slack in the system has now been used up. A most valuable asset, and one Page also 
claims for the Cumulative Sum chart, is that there is no cut and dry set of rules that 
can be applied by rote. It is required that one think about the particular problem he 
has at hand, and this should lead to a better solution than can be had any other way. 
Another most important feature of the Acceptance Control system is that the control 
limits are set up after consideration of the process requirements called for by the 
specifications. Thus the problem of tight-control can be avoided when such control is 
not required. 


In summary, two new methods of process control have been proposed recently. 
One, the Cumulative Sum chart (or the Geometric Moving Average chart) is designed to 
furnish a more powerful tool for detecting the small process shifts that are a serious 
nuisance when specifications are tight and variability must be held to an absolute 
minimum. The second, the Acceptance Control chart, is designed to provide a more 
efficient tool for situations in which normal control or more relaxed control systems 
are desirable. 

Both of these approaches would seem to merit consideration as auxiliary tools in 
the kit of every Quality Engineer. Indeed, it would not be surprising to see them re- 
place the Shewhart chart completely since each is more efficient in its own area, and 
the respective areas encompass the whole field. 


Tlustrative Example 


Table I shows a set of twenty averages of four observations per subgroup. The 
first ten averages come from a random, normal distribution having M =0, 7 =2, 
TY =1. It is assumed that a shift in process level of + 3/40, (i.e. + 1.5), takes 


place between the tenth and eleventh samples so that the second ten averages come from 


a random, normal distribution having =1.5, T =2, he 


The data in Table I are plotted on a Shewhart chart, a Cumulative Sum chart, a 
Geometric Moving Average chart and an Acceptance Control chart in Figures 1 through 4 
respectively. This is done for illustrative purposes only and is not intended as a 
proof of the superiority of any system. Only one type of process shift, as well as 
only one particular level of shift, is even considered and only one set of control 
criteria is looked at for each system. Furthermore, the objectives of the Shewhart, 
Cumulative Sum and Geometric Moving Average charts differ from those of the Acceptance 
Control chart. The first three all assume that the process should be held as closely 
as possible to the nominal value, zero. The Acceptance Control system illustrated here 
assumes that the specification limits are at + 50, (+10), rather than at the +30, 
(+6), natural process limits, or perhaps even tighter ones, for the other systems. 

The objective of the first three systems is to detect any process shift ss quickly as 
possible. The Acceptance Control chart is designed to detect shifts as large as the 
Rejectable Process Level (R.P.L.) as quickly as possible while simultaneously trying to 
avoid process adjustments for shifts as small as the Acceptable Process Level (A.PeLe)- 


The values plotted on the Shewhart chart, Figure 1, are the subgroup averages, X- 
The control limits are 0 + 30; =+3.0. All twenty points are plotted because without 


any supplementary run procedures, the chart does not indicate any shift in process 
level. It would depend on which run procedure was specified as to when (or whether) 
the process shift is detected. For example, if 0 + 27 — were set up as warning lines, 
adjustment might be called for at time 19 since two suctessive points lie between the 
warning and action lines. 


The values plotted on the Cumulative Sum chart, Figure 2, are the cumulative sums 
of the deviations of the subgroup averages from the nominal value, | - 0). Tnese 
values are listed in Table I. The contro] limits are a V-shaped mask with the lead 
distance d = 5 and the tangent of half the angle of the V, tan 9 = 0.35. 
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This particular mask was selected because it has an « risk of about 0.3%, as do the 
3 0 limits for the Shewhart chart. Only the first fifteen points are plotted since a 


point falls under the V-shaped mask when it is used for point 15. Thus, process ad- 
justment would have been called for at this time. 


The values plotted on the Geometric Moving chart, Figure 3, are the weighted 
averages, Z, = + The weighting constant, r, was chosen to be 1/he 


These values are also given in Table I. The control limits are 0 + 3 To» where 


c, = a - and they equal +1.13. Only the first eighteen points are plotted 
since the eighteenth point exceeds the limit and calls for a process adjustment. 

The values plotted on the Acceptance Control chart, Figure 4, are the subgroup 
averages X- With the specification limits set at + 10.0, it is decided to select as 
the Rejectable Process Level + 6.0. (This could have been an arbitrary choice, or it 
could have been done on a basis that a process centered at 6.0 would have less than 
24% of its individual items beyond 10.0.) Since the sample size has been specified as 
four, the Acceptance Control Limit (A.C.L.) = + 4.35 if we want to limit our B risk to 
5%. The Acceptable Process Level would be 1.57 if we want % = 0.3% or 2.70 for an 
% = 5%. All twenty points are plotted here since none exceed the control limits; nor 
should they under the above definitions of the control requiruments. 


TABLE I 
- 0) = rk, + (1-r)%_33 r = 1/4 
1 1.6 1.6 0.40 
2 O04 2.0 0.40 
3 -1.1 0.9 0.02 
4 -0.7 0.2 -0.16 
5 1.6 1.8 0.28 
6 1.9 0.24 
7 1.4 0.5 -0.17 
8 0.8 1.3 0.07 
9 0.2 1.1 0.00 
10 1.4 0.3 -0.35 
04 O.1 -0.16 
12 2.1 2.2 0.40 
13 2.0 0.25 
14 24 eh 0.79 
15 1.9 6.3 1.07 
16 1.1 Tok 1.07 
17 0.1 723 0.78 
18 24 967 1.18 
19 2.9 12.6 1.61 
20 14 14.0 1.56 
x3 Average of 4 observations per subgroup taken at time t. 


Subgroups 1-10: Random Normal Distribution, M=0, J =2, oS =1 
Subgroups 11-20: Random Normal Distribution, =1.5, 0 =2, =1 
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STATISTICAL METHODS FOR CERAMIC 
PROCESS CONTROL AND EXPERIMENT PLANNING 


R. S. Bingham, Jr., Manager, and J. L. Gioele, Senior Engineer 
Quality Control Department, The Carborundum Company 
Niagara Falls, New York 


Introduction 

S paper aims to provide ceramic engineers and technologists with insight into 
ways to profit from the use of statistical principles in resolving process control and 
development problems, The general principles are reviewed, Their use is demonstrated 
with examples from a glass-to-metal seals plant and a laboratory search for optimum 
time, temperature and pressure conditions to obtain maximum density in pressure sinter- 
ing of alumina. 


A number of references to the general quality control literature are cited to sup- 
plement the limited number of papers available from the ceramic literature, These gen- 
eral papers include specific references to problems of ceramic interest.** As wili be 
evident to the careful reader, the statistical principles are identical regardless of 
the industry to which applied. Their successful use depends on the ingenuity of the us- 
er in recognizing the appropriate statistical counterpart to each ceramic problem, It 
is emphasized that most quality control procedures can be readily learned and require 
little more mathematics than arithmetic. Thus the methods are suited to a one-man plant 
technical department as well as to larger plant or staff organizations. 


Statistical consideration of a ceramic problem is apt to concern itself most fre- 
quently with two properties: (1) the level of a process or product quality (such as the 
average moisture content of a mix), and (2) the uniformity of the process or property 
(e.g. as measured by differences in moisture content in several samples from different 
parts of the mix.) Estimates of level and uniformity based on measurements or judgments 
made on small samples of product can be used to predict limits within which additional 
sample measurements would fall, or if a process is concerned, limits which will contain 
a selected percentage of future measurements as long as the process remains stable, A 
greater understanding of variability, how to calculate and judge it, can help ceramists 
resolve many recurring problems. 


Process Control Uses of Statistics 

How should a ceramic process be controlled? It is reasonably clear that the com- 
position must first be chosen to meet the final product property specifications, Then, 
the raw materials must be selected, The process by which the ware will be made and 
fired is the next choice, (This includes conditions for drying, glazing, etc.) If di- 
mensional requirements are important, dies or molds must be made to a size to account 
for within-process shrinkage. However, it may not be obvious that designing the prod- 
uct, choosing the process and inspecting the final product does not represent "control" 
nor does it necessarily guarantee production of a single saleable piece. Not only must 
the equipment be tuned to the proper settings, but also, if operators are expected to 
exercise judgment or adjust the equipment they must be given ways to verify the action 
they have taken. 


In the press of business, it is not uncommon temporarily to depart from existing 
specifications to obtain volume production with the expectation of paying the price in 
off-grade ware at a later stage. Lacking lete knowledge, the gamble is that the a- 
mount of off-grade ware will be less than production increase. The successful man- 
ager will have built into the process only enough check points (and these at key posi- 
tions) to satisfy himself that meaningful in-process specifications are being met along 


* Reprinted in part with permission of the Journal of Canadian Ceramic Society. 
(See Vol. 28, 1959) 


** For example, Section 11 of (1) discusses a wide variety of applications in 
the chemical industry and contains hundreds of references. (2) indicates prob- 
lems of interest in research, development, process control and control labor- 
atory areas. Appropriate statistical tools are suggested for each problem. 
Many of these are of direct interest to ceramic engineers. (3) reviews batch 
control and illustrates with control charts and multiple regression procedures. 
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the way. The heart of the problem, of course, is in picking the places for checking 
and in setting the specifications at each point. Fundamental to both of these consid- 
erations is the ability to measure* the process at each point, and to know what correc- 
tive action mst be taken to restore the process to desired values,** 


These are the established elements of control. What then do statistical methods 
contribute? Very briefly, they provide methods for setting specifications and judg- 
ing how well a process meets them. One of their concepts asserts that after all "as- 
signable" (or identifiable) causes have been removed, only random variation will ex- 
ist.#+ Limits can be determined which distinguish between the effect of real factors 
and this chance or "residual" variation. The statistical techniques can also be used 
to determine how well various steps in the process influence final product properties 
toward selecting key control points, From a sound theoretical basis, these methods can 
describe the frequency and number of samples necessary for economic detection of condi- 
tions which would lead to off-grade ware. In inspection of product, decision methods 
are availatle for comparing the number of defects, per cent "chipped," "“off~color", 
etc., on one day's operation with another. Such measures are valuable in assessing 
formulation or process changes, 


Statistical Aids in seperate Tests 
n process development, experiments can be planned using statistical "designs" 


which permit the simultaneous examination of the effect of many process variables on 
intermediate or final product properties. One such example might study the effects of 
drying rate, position in the dryer, drying temperature, and initial moisture content on 
shrinkage and warpage. Not only do such planned experiments provide pr 
of experimentation, but also they answer questions about the joint effects of the var- 
iables that "one-factor-at-a-time" tests do not consider. From such test results mini- 
mum risk decisions can be made concerning which variables control in complex situations. 
In ceramic formlations, where multicomponent systems are common, "factorial" and other 
experimental designs can give quick insight into the response of a property to composi- 
tion change much as a phase diagram does ,*HHHt 


Bulk Material Problems 

mater intermediates used by the ceramist are frequently bulk materials 
rather than discrete units. Fortunately, the same general principles of sampling apply 
regardless of the way materials may be handled. To characterize a "batch" or "lot" re- 
quires a knowledge of variation within the lot and types of stratification that occur. 
In addition, the variation of the test method must be considered in specifying a plan 
for control or accpetance of bulk materials, The use of experimental designs to obtain 
basic design data for such plans is described later in this paper. References (13, 17) 
provide additional general information and (5) examples of application in white alumina. 


The following examples illustrate the use of statistical methods in two different 
ceramic areas - process control and research, As will be evident, statistical proce- 
dures can provide the setting for imaginative scientific and engineering identification 
and solution of problems. Frequently, the strategies used in bringing about the final 
change are more important to the total solution than all the detailed investigation 
preceding it. 


# (22) discusses specific gravity measurements in whiteware. 


## Reference (l;) discusses this in greater detail. (16) describes non- 
statistical considerations in Slip House control. 


#“## See reference (1) for general discussions on quality control including 
setting specification and "natural" process tolerances. 


ste See reference (9) for details for making control charts for both attri- 
butes and variables data. (19) is the original source for control chart 
principles including many examples and empirical verification for the 
procedures. (12) gives ceramic examples of confidence limits, 


suet (11) discusses the application of one such experimental design to white- 
ware problems, (6,7,8) offer detailed instructions in the procedures. 


set (15, 18) provide interesting insight into problems in human relations 
common to plant problems and studies of this type. 


APPLICATIONS IN A GLASS-TO-METAL SEALS PLANT 


A sensitive plant management quickly detects deterioration in its product quality 
from customer complaints, scrap rise or in-process rejections as well as an unfavorable 
balance sheet. It also recognizes that if its regular control procedures were in force, 
something more fundamental will be required to correct the situation. In the following 
example, management of the Latrobe Plant had (1) decided that action was needed, (2) 
roughly surveyed and identified the product line to be considered, (3) set loss reduc- 
tion as primary objective, and () called for a concerted effort. Two quality control 
engineers from the central staff were called in to assist the plant product engineering 
group. 


A — to Introduc: S.Q.C. 

an t meeting with production and research supervision to review 
the en the stated objectives, and a brief summary of how statistical quality con- 
trol methods work, a team was formed to conduct a process survey. The seals production 
foreman, a development engineer and two quality control engineers formed the team, 
Members of the team were selected in anticipation of skills that would be needed to 
solve the problem.* A team was chosen rather than individual effort since the quality 
control engineers were not conversant with the seals process nor the plant procedures. 
Also, it was expected that the production foreman could arrange for special samples, 
inspection, modification of operating procedures, etc., as needed. The research engi- 
neer provided technical know-how concerning the product design, raw material properties 
and significant process controls. 


The process survey had 3 purposes, to: 


(1) establish the existing process and its controls; 

(2) determine the quality control needs of the process; 

(3) identify the present quality level, factors contributing to it, 
and to measure the state of statistical control present. 


The initial survey was limited to raw material receiving, mixing, pressing, sintering, 
inspection, packaging and shipping. (See Figure 1). 


An analysis of inspection GLASS wiGs KOVAR 
losses pinpointed the areas suggest- weomine 
ing most profitable study. Further _ [ 


examination showed that a few pro- 
duction items accounted for a 
major part of the total loss, To 
maximize team effectiveness and 
produce the most results in the 
brief time allocated, the team ef- 
fort was directed first toward 
these few high loss items. The 
program would be extended to the 
other areas and items as the need 
arose. The following discussion 
covers many of the important areas 
in detail. 


Raw Material 

The manufacture of glass 
seals consists primarily of as- 
sembling various components such 
as leads, glass beads, eyelets, 
headers, etc., by fusion and/or 
welding into a finished unit. 
(See Figure 2.) 


* See (1) for a more de- é 
tailed discussion of Figure l-GLASS-TO-METAL SEALS PROCESS FLOW SHEET 
Carborundum's team 
approaches. 
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GLASS BEAD ASSEMBLY The team set out to answer the ques- 
ees Caer tion, "How well were the components sup- 
plied by our vendors conforming to blue- 

LEAD print specifications?" Random samples were 


selected and measurements were taken rela- 
yNEADER tive to the specifications. The distribu- 
Gh { tion of the length of 278 leads randomly 
"\evass seao selected from stock is shown in Figure 3*. 
Table I illustrates the calculation to com- 
pare performance with specifications. 


L * LENGTH (CONTROLLED DIMENSION) 


Figure 2-GLASS-TO-METAL SEALING UNIT 
* Note how dramatically these simple graphs demonstrate vendor performance. 
This makes them valuable in discussing quality improvement. (The nominal 
value is 1.000+ reading shown, i.e., 1.053 etc.) 


[_] WITHIN SPECIFICATIONS 


OUTSIDE OF SPECIFICATIONS 
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240 | 
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LENGTH - INCHES 
Figure 3-HISTOGRAM SHOWING LENGTH DISTRIBUTION OF 278 LEADS 


TABLE I - COMPUTATION OF AVERAGE AND STANDARD DEVIATION 


Lead Length Fre- 
2055 15 3 135 Notation: 
31 -2 62 12 = average 
057 -46 46 = standard deviation 
2058=(A) 60 c = class interval = .00l1 
2059 53 +1 +53 53 A = arbitrary midpoint 
2060 3h +2 +68 136 u = deviation in cells from A 
2061 20 +3 +60 180 (x = A + cu) 
3 7 % 
+ 
Calculations: 
c= Sf = 270 Sfu = 39 = 989 F= .058 2001 20581" 
Oy = /278 = .00188" 
Per Cent of Parts Specs (Normal Curve Theo 


(a) Below lower spec: (.054 = .0581)/,00188 = -2.18 or 1.5% below 1.05)" 
(b) Above upper spec: (.059 - .0581)/.00188 = +0.48 or 31.6% above 1.059" 
where (spec - ¥)/o, =z compare z with Normal Curve Area*# 

#Add 1,000 in. to get actual values. ** See reference (9). 


(Length of 278 Leads Randomly Selected from Stock) 
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Toward improving incoming inspection, operating characteristic ("o.c.") curves 
such as shown in Figure were prepared to show plant inspectors the probability of 
accepting lots containing a certain percentage of nonspecification parts for a given 
seole size and acceptance number, Note that if protection is wanted against accept- 
ing lots containing 5% defective parts, the plan offering the greatest protection is 
the one which calls for a sample size of 75 and acceptance number of 0.* Table.II 
shows that the greatest incremental increase in protection occurs when going from a 
sample size of 25 to a size of 50. Also that a sample size of 25 and acceptance num- 
ber of O offers the same protection as a plan calling for a sample of 50 and accept- 
ance number of 1, 


o 


ce 


LOT % DEFECTIVE LOT DEFECTIVE LOT % DEFECTIVE 


' 4-COMPARISON OF OPERATING CHARACTERISTIC CURVES FOR 
DIFFERENT SAMPLE SIZE SAME ACCEPTANCE NUMBERS 


TABLE II - CHARACTERISTICS OF SAMPLING PLANS 


Probability (%) of 
Sample Acceptance Accepting lots 5% 
OC Curve Size er Defective 
I 25 fe) 28 
i 65 
Ir 50 8 
1 28 
III 75 0 2 
1 12 


ther study involved the control of physical dimensions of pressed small glass 
beads. (These glass beads are an essential part of glass-to-metal sealing units used 
in the manufacture of transistor assemblies. See Figure 2.) 


The goal was to produce uniform length beads as too much glass or not enough glass 
resulted in a defective seal. Green length and green weight were chosen for control. 
(See Figure 5.) 


* "acceptance number = 0" means, accept the lot as adequate quality if 
no defective pieces are found in the sample. If 1 or more are found 
reject the lot. 


2 C “ACCEPTANCE NUMBER 
80 (SINGLE SAMPLE) ee 
N=25 N=75 
20 
= 
itd 
jae 
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= 


Figure 5- CONTROL CHARTS FOR GREEN LENGTH & GREEN WEIGHT-GLASS BEAD 


The average, range, and control limits were computed for samples of 10 measure- 

ments for green length from existing data. (May 27 to August 5.) It was evident that 

the was out of control for length. There were no back data available for green 

= 

1 On August 12, random samples of 5 beads were taken directly from production and 
checked for green length and green weight. New control limits were calculated (August 

August 16.) Points representing a change in the process were labeled and 

attempts made to assign causes for lack of control, 


The chart indicated that green length was still out of control, and that the green 
weight was steadily increasing and also out of control. The team decided that the 
press was in bad condition and that repair was imperative before adjustments could 
effectively be made to bring the process under statistical control. The press was sent 
to the maintenance shop to be overhauled. On August 20, the press was put back into 
operation but the level of the process was as variable and out of control as ever. 


Close examination of the charts showed that directly after overhauling the press 

(August 20 to August 26), two significant changes had occurred. A decrease in varia- 
tion from bead to bead was evident from the reduction in the range. However, the fre- 

: quency of punch breaking and bending seemed greater. A closer look at the press showed 
that a bottom bushing (which permitted excessive play in the die, thus causing the 
punch to bend and break) was missing. This condition was unknown to anyone and it is 
doubtful if it would have been discovered without the aid of the control chart. The 
bushing was replaced on August 28 and the process was brought under statistical control, 
The reject rate was reduced from 18% to 2%. 


Jigs 
Discussion so far has been limited to the control of defects attributed to raw ma- 
terial and process variables. However, many of the defects could also be attributed to 


| 
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equipment design. In particular, an analysis of scrap causes revealed that approxim- 
ately 75% of the total scrap was due to the jigs. 


At a meeting of the team (with manufacturing engineering) it was decided to con- 
centrate on improving the jig design. 


Figure 6 depicts the results, The particular part chosen for the study was run- 
ning at a 38% defec: level prior to the change in jig design. With the adoption of an 
improved, more efficient jig, the scrap level was reduced to 7%. As a result of this 
quality, 100% gave way to sampling inspection with an additional 
saving in inspection cost and scrap. 

100} NOTE: PRIOR TO THE USE OF THE NEW JIG 


PRODUCTION WAS INSPECTED 100 
L AFTER THE NEW 
PRODUCTION 


S SAMPLE INSPECTED. 


LEVEL 36% 


Figure 6-PERCENT DEFECTIVE CHART 


The contribution of the team to the solution of the problem was mainly to call 
attention to the cause of the difficulty. The modification which the redesign entailed 
required 


many hours of work by the engineering group. 


Final ion 

ion was principally used to inspect the final parts. This method 
relied heavily on the use of judgment by the inspectors in accepting or rejecting fin- 
ished lots of seals ready for shipment. To test the effectiveness of this inspection, 
the team submitted lots of finished parts which had been inspected by the day shift in- 
spectors to the night shift inspectors for reinspection, and vice versa, To eliminate 
the possibility of any bias in the checks, the inspection department was not alerted to 
what was being done. 


The results showed that under the system being used, it was impossible to deter- 
mine a realistic outgoing quality level on the item under investigation because no two 
inspectors inspected~walike, Te yaa not Tar te reject material 

epted by ther inspector. 


This situation called for a complete review of the classification of defects. 
Customers were visited to obtain concise definitions of the quality characteristics re- 
quired, It is interesting to note that in some cases final inspection was "too tight" 
on certain quality characteristics of the finished part. Consequently, parts that 

were considered good and usable by the customer, were being rejected by in- 
spection. Based upon the response from customers, the definitions of quality defects 
were revised and new standards set up. The retraining of inspectors to the use of the 
standards and new definitions of quality defects led to immediate improvement and addi- 
tional loss reduction, the objective of the work. 


In summary, this example has shown the effectiveness of modern statistical quality 
control principles when applied in an orderly, systematic investigation of ceramic pro- 
cessing problems, These methods afford economically controlled material from entrance 
to exit. Quantitative, rather that qualitative judgment makes routine comparisons more 
discriminating even though attributes are being controlled. Graphical devices, such as 


CED 
QUALITY LEVEL CHANGE “a 
: DUE TO NEW JIG 
i 
20 EVEL 7% 
° 
25 % 35 
DAYS 1 
3 
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control charts, histograms, and sampling table consequences can instill quality minded- 
ness in the operator. The team approach can make the most of many people's experience 
while directing concerted effort to the most pressing problem. Only the most recalci- 
trant can be opposed to the recommendation for process improvement which his work has 
helped to prove essential. These brief examples may suggest that results are always 
forthcoming rapidly. In several instances, solutions were only found after endless 
hours of engineering work by other plant personnel, 


Another problem lending itself to systematic examination, is that of assessing 
what combination of levels of each of a number of process factors will produce maximum 
yield (or minimum cost; etc.). The next example discusses these approaches. 


LABORATORY SEARCH FOR MAXIMUM DENSITY 


research into the mechanism underlying pressure sintering it was of inter- 
est to find combinations of time, temperature and pressure which would lead to maximum 
density.* The theoretical pressed density for the alumina to be studied was 3.98 g/cc. 


Selecting the Model and Experimental Design 

Preliminary work suggested the ranges shown in Table III within which further ex- 
periments should be performed. Lacking more complete knowledge, a second order math- 
ematical model was chosen to represent the influence of each of the variables on 
pressed density. The initial model expressing density as a function of pressure, tem- 
perature and time was: 


d= bo + + boxy + byxy + + bogey? + 


(1) 


where x, = (pressure - 1600) /370 (2) 
Xp = (temperature - 150) /150 
X3 s (time - 20.0) /6 
d = pressed density (g/cc) 
The scaling of equation (2) was designed to produce equal changes in density for a 1 
unit change in each x. 


TABLE III - RANGE OF EXPERIMENTAL VARIABLES 


Pressure Temperature Time 
(psi) (°c) (min. ) 
1000 1205 10.2 
2200 1695 29.8 


Since the objectives included learning as much as possible about the time-tempera- 
ture-pressure system, an experimental design was chosen which permitted "mapping the 
surface" of the system.** The properties of the design allowed the experimental work 
to be carried out on 3 separate days ("groups" I, II, and III) while still retaining 
the ability to separately assess the influence of each of the variables independently. 
Also, each group had a built-in measure of experimental error for control. Finally, 
from the overall design a test could be made later to judge whether the model chosen 
was adequate. Table IV shows the experiments performed and the results. 


* Experimental details and a discussion of their significance has been 
reported separately in a paper by the researchers, G. Mangsen, 
W. Lambertson, and B. Best.: Journal American Ceramic Society 
Vol. 43, No. 2, February 1960 p. 55 - 59, "Hot Pressing of Aluminum 
Oxide". 


#*# See (11) for another application in studying whitewares, Also 
(6, 7, 8, 10, 20) for other statistical designs. 
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TABLE IV - TEST SEQUENCE AND RESULTS 


Experimental Conditions Results 
Group Run Design Units pressure temp. time Fired Density | 
No. No. x, X5 x3 (psi) (°c) (min. ) g/cc) 
3 1 +1 -1 +1 1970 1300 26 2.17 
2 +1 +1 -1 1970 1600 14 3.69 
3 fe) fe) 1600 1450 20 2.92 
h = +1 +1 1230 1600 26 3.53 
5 -i -1 «1 1230 1300 uy 1.77 
6 1600 1450 20 2.92 
II 7 -1 -1 +1 1230 1300 26 1.77 
8 1600 1450 20 2.88 
9 -1 +1 -1 1230 1600 iy 3-37 
10 0 fe) 1450 20 2.91 
1 +1 +1 +1 1970 1600 26 3.83 
12 +1 -1 -1 1970 1300 4 2.10 
III 13 1.63 0 ft) 2200 1450 20 3.05 
0 1600 1450 20 2292 
15 -1.63 fe) 1000 1450 20 2035 
16 1.63 1600 1450 29.8 3.04 
17 ie) -1.63 fe) 1600 1205 20 1.66 
18 1600 1450 20 2.87 
19 1.63 1600 1695 20 3.86 
20 fe) te) -1.63 1600 1450 10.2 2.61 
Data is and Interpretation 


on ©. ese results shown in Figure 7 (summarized in Table V) 


> 
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Figure 7- EXPERIMENTAL RESULTS- DENSITY AS A FUNCTION 
OF TIME, TEMPERATURE AND PRESSURE 
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TABLE V = CHANGE IN DENSITY = ESTIMATES OF THE 
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EFFECT OF VARIABLES 


Change in Density 
Variable Per Design Unit Per Experimental Unit Units 
(g/cc/unit) 
time 2018 - 2075 230 = 1.25 g/cc/100 min. 
temperature 257 g/cc/100° c, 
pressure g/cc/1000 psi 


suggested that time had slight effect at the low levels 


about two times more pronounced effect at higher levels. 


to be about eighteen times more influential than time, 
density was intermediate between temperature and time. 
more complete form with their 95% confidence limits.* 


of temperature and pressure but 
Even so, temperature seemed 

The influence of pressure on 

Table VI shows these results in 


TABLE VI - COEFFICIENTS AND THEIR ERRORS* 


Symbol Pressure Temperature Time Value of Confidence 
(x,) (xp) (x3) Slope Limits 

by x .018 
bo x 2765 = .018 
b3 x 2080 + .018 
x x 006 + 
bo3 x x 2029 = 02 
b33 -.021 + .018 


* Slopes have units "g/cc per design unit". 
mental units divide by 370, 150, and 6 res 
temperature, and time. (See equation (2). 


2 2 2 
** Coefficients of x, -squared; x,°, 


center point. 


#e# The 95% confidence limits shown are based on the degrees of 
freedom of the experimental error estimated from the replicated 


To convert to experi- 
tively for pressure, 


Table VII provides an alternative way to assess the significance of the various coeffi- 
cients. From analysis of these tables, it was concluded that: 


(a) in the model, both the first order (x, )and 
were needed. 


second order (x,”) terms 


(b) the interaction terms (x4x5) had small effect. 
(c) no substantial difference occurred from groups I and II to III. 
This meant that any changes in apparatus, procedure or material 


had no influence on the results, 


(d) experimental control appeared excellent - the measure of exper- 


imental error*++ in groups I and II was not 


different from that 


in group III, and also compared well with a previous estimate 


based on 7 separate runs. 


(e) a better model should be sought. Although more than 99% of the 
density variation had been accounted for, based on this experi- 


mental error, the "lack of fit" term was si 


gnificant. 


* Values are judged ened si icant if their ranges do not in- 
) is 


clude zero. 


Hence, 
is not. 23 


+02 


significant but +.02h) 


+ (how well repeated runs made wy one set of conditions agreed - 0, 


» O design units were used 
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TABLE VII = ANALYSIS OF VARIANCE 


Source Degrees of Sums of Mean 
Freedom Squares Squares 
Constant 1 158.034 
First Order 3 8.3592 2.786% 
Second Order 6 209k 20157# 
Blocks 1 20030 
Residual 9 21082 
lack of Fit (5) (.1059) 20212 
Experimental Error (4) (.0023) 200058 
Total 20 166.5993 
* significant at better than 0.5%; i.e., chance occurrence 
of exceeding the "critical mean squares" based on an exper- 
imental error of .00058 is only 5 in 1000. 


Clues for a Better Model 

© more lerstand the implications of these results a "canonical analysis" 
was made of the equation. This procedure simplified equation (1) by rotating and 
translating its axes. The resultant equation 


- 7.66 = (3) 
where = )+.157(x-9.58) 
= -,001( )+h21( )+.907( ) 


defines a set of ellipsoids. A comparison of the coefficients of Y; with their 95% 
confidence limits + .018) suggests that the effect of Y; can be ignored for purposes of 
further examination. Thus, equation (3) reduces to a series of ellipses* to represent 
the time-temperaure-pressure system. 


In the course of finding equation (3), equation (1) with appropriate coefficients 
of Table VI was differentiated partially with respect to x), 2», and x3, the three re- 
sulting equations set equal to zero and simultaneously solved for the maximum, A max- 
imum density of 7.66 g/cc was indicated for x, = 0.672, = 11.28, and x, = 9.58 
(design units for pressure, temperature and time, respectively.) This in contrast 
to the maximum theoretical density of 3.98 g/cc. Since the polynomial model chosen had 
its coefficients determined in the experimental space of interest, its results should 
not be extrapolated too far beyond it.** Table VIII shows the path of steepest ascent 
from the center of the design toward theoretical density based on the "first order" 
terms only and indicated that close approximation to theoretical density could be 
achieved within the space explored, Figure 8 shows typical constant density contours 
calculated from the full model for one selected time (26 min.). The agreement with the 
experimental points is remarkably good. Examination of similar diagrams for other time 
values further shows that its effect on density was slight in these pressure-tempera- 
ture ranges. 


The procedure described is only one of the many ways that experimental designs can 
be used to help experimenters. In this example, the design and accompanying statistic- 
al analysis aided the experimenter to achieve quantified, meaningful results with a 


* The system has in effect been reduced to 2 variables, d = f(Y),Y,). 
However, and are functions of x pressure, temperature 
and time.) This means that 2 new vartab can be found which more 
fundamentally represent the physical effect of pressure, temperature 
and time. The coefficients of equation () should aid the experi- 
menter in selecting such variables. 


+ A better "fitting" model could be expected to predict a maximum value 
closer to 3.98 g/cc. 
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TABLE VIII - PATH OF STEEPEST ASCENT 


level of Variables* 


Pressure (xj) Temperature (x5) Time (x3) Calculated Density 
Ue | De. ue Ee ue| De ue Be ue} De. ue 

1600 20 1450 0 20.0 2.90 

1627 2073 1495 0.3 20.2 | .032 3.15 

1654 2147 1540 | 0.6 20. 2063 3.39% 
1681 2220 1585 0.9 20.6 | .09h 3.63 

1709 2293 1630 1.2 20.8 | .126 3.87 

1736 2367 1675 1.5 21.0 | .158 Ls 


* E. ue = Experimental units, D. u. = Design units. 


#* The full second order 
for the 3.39 and .11 g/cc shown. 


tion predicted 3.1 and 4.02 respectively 


minimum of experimentation. 


TIME = 26 MIN. 


MEY 
4 - EXPERIMENTAL PT 


Figure 8- CALCULATED DENSITY CONTOURS 


In addition, the derived mathematical model permitted 
"desk-side" examination and study thus stimulating modification and improvement of the 
~- model.* In the final test, the organization of an experimental program has been opti- 


mum when it maximizes the inventiveness of the scientist in formating and testing 
his theories-- mathematics provides insight, generality and extension. 


The work reported in this paper is properly that of many members of The Carborun- 
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AN INTRODUCTION TO CUMULATIVE SUM CHARTS 


H. Mack Truax 
Atlas Powder Company 
Wilmington, Delaware 


Last year an article in the Chemical and Engineering News! indicated 
Atlas Powder Company's successful application of a new statistical qual- 
ity control approach. This new approach centers around the use of the 
cumulative sum or CS chart. This chage was originally devised by Page® ,8 
and further expounded upon by Barnard“ during the last six years. 


The purpose of this article is to first show that cumulative sums 
are merely a logical extension of regular charting techniques and then 
to describe the actual use of CS charts in: practical application. 


In discussing applications, two major advantages will be stressed: 


1) Quicker detection power when a true shift from control 
occurs. 


2) The ability to use a look-and-see approach that allows 
the application of graphical estimation techniques. 


It will initially be assumed that the data (x measurements) are 
equally spaced and are independently and normally distributed with con- 
stant system noise or variance o*, and that detection or estimation of 
location parameter shifts is a primary interest. 


Control chart strategy will be considered first and then Page's ex- 
tension of regular control charts to the cumulative sum concept will be 
shown. 


Strategy 


An important general characteristic of any control chart is the av- 
erage run lengths associated with the scheme. When the true process level 
remains constant, the average run length, L, of the scheme is the expected 
number of samples recorded before action is indicated. Now L is a func- 
tion of the true process level relative to the control level. The crite- 
rion for selection of one control scheme over another should be either 


Max Lo for fixed Lj, or 
Min Lj for fixed Lo 


The average run length is Lg when the process level is at the con- 
trol point and is Lj when the process has shifted from control by some 
magnitude A. Lo and Lj are functionally related (sometimes in a compli- 
cated manner) to the usual Type I and Type II errors. 


It has been shown’ that for fixed Lo, L, can be reduced on the regu- 
lar Shewhart? chart by the addition of warning lines such as is shown in 
Figure 1. This extension of a regular control chart has not only the 
usual action lines but also has warning lines between the action lines 
and the control point. The decision rules are as follows: 


1) If a single point is outside an action line, then 
action is indicated. 


2) If two consecutive points are between the action and 
warning lines and on the same side of the control point, 
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then action is again indicated. 


3) If other sequences of pcints occur no action is 
taken on the system. 


The Transition 


This regular control chart can now be reformulated into a cumulative 
sum chart. In Table I, the different zones are given numerical scores. 
A score is applied to each of the points on a regular chart, based upon 
the zone in which the point falls. A series of such scores, yj, is also 
shown in Table I. These scores are recorded and a cumulative sum is for- 
med by adding each value to the last previous sum of values 


n 


Sp = 2 


It is assumed no action has beer taken since yg. The CS decision 
rule consists of taking action when 


For this particular scoring system h = 2. In the example shown, in 
Figure 2, the points (S,, n) are plotted on a CS chart and a special mask 
constructed with a vertical slot of 2 units. The mask may be applied by 
placing the upper left corner of the mask on the last point, S,. The ac- 
tion condition, h, will only be satisfied when (Min S;) either touches or 
falls beneath the mask. 


Due to our transition method this CS procedure is now a one-sided 
test. However, it does show the degenerate case of cumulative sums in 
which only individual samples and two consecutive samples are considered. 
Although this CS procedure offers no run length benefits over the previais 
regular chart, it does show that a CS procedure can consider consecutive 
runs of points through a single action rule. 


Additional zones could be established on the Shewhart chart which 
would allow consideration of more complicated cases of runs up to k con- 
secutive points. These could in turn be transformed into a CS procedure. 
Although the average run length should be improved by such techniques it 
now becomes obvious that a CS chart of the original xj data should offer 
further benefits. There are now no a priori restrictions on the number 
of consecutive points that can be considered and the original xj data 
should offer more information than would the yy scores. 


CS Control Concepts 


The one-sided CS chart for original data has physical characteris- 
tics similar to the discrete cgse for scores. Optimum average run length 
properties for CS charts occur’ when 


n 
Sn = = (x;-k) 
i=l 


Uptu 
ko 


2 


Here ug is the control level and u,, the desired rejection level. Once 
again, as in the discrete case, the action criterion becomes 
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and a mask of the same general form as previously shown for a one-sided 
test in Figure 2 could be used for taking action. A disuccion by Ewan 
and Kemp* of the one-sided test, complete with tables and formulaes, ap- 
pears in Biometrika. 


The two-sided CS test can now be discussed since the relationship 
between regular and CS charts has been indicated. This is the more im- 
portant case because many charting procedures in processing industries 
require two-sided control. 


In this case S, is defined as 


n 
S, = = (xq - k) k = up. 
0 


Action is now based on a V-shaped mask. The vertex of the V points hori- 
zontaily forward when placed on the chart. Note in Figure 3 that there 
is a horizontal distance D which has not been cut from the mask. There- 
fore, the important parameters of this mask are the distance D and the 
slope M. The point of the mask indicated by the arrow is placed at the 
last cumulative sum and past data is reviewed. As with earlier masks no 
action is taken if all points since the last action point are visible. 
The action signal results when some S; in this interval either touches or 
falls behind the mask. 


Two points in Figure 3 have been illustrated as larger dots. When 
the mask was placed on the second of these points, action was indicated 
by the first point falling behind the mask. Therefore, when reviewing 
future data, only points after this second larger point are considered. 
This procedure assumes that corrective action was taken at the time of 
the action signal and system response lag was trivial. 


Masks may be constructed by two techniques. One solution requires a 
“cut-and-try" approach whereby D and M are selected by analysis of past 
CS charts. In practice a series of CS charts are prepared. Known pro- 
cess operational changes and product quality information are also noted 
on these charts. A process engineer familiar with the system is given 
the task of preparing a "best"' V-shaped mask. He is instructed to select 
the final mask so that the cumulative sums remain within the V during 
periods when the system appears to be operating in a satisfactory manner. 
However, during periods when system changes were known to have occurred 
or unsatisfactory quality existed, his mask should be designed to detect 
these situations quickly. The other approach requires approximations to 
the solutions of the random walk with absorbing barriers on the edge of 
the V. These solutions allow the determination of D and M based on fix- 
ing Lo, L,, plus the aageitude of process shift that must be detected. 
Goldsmith and Whitfield? present charts for use in the selection of D and 
M, and show that the run lengths of these CS procedures are not seriously 
affected by first-order serial correlations except for negative correla- 
tions near -1. This information is very important, since experience has 
indicated that many industrial systems exhibit some degree of serial cor- 
relations. 


An actual example can best show the advantages of a CS approach. 
Both V-mask construction techniques were applied. The solutions to the 
two schemes are shown in Table 2. In the mathematical solution was 
selected to correspond to the same average run length as the previous 
Shewhart contrel chart in use. The “cut-and-try" method showed good 
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agreement between the two approaches. 


The average run lengths for this example are shown in Figure 4. All 
examples are two-sided (based on x data). This clearly shows the improed 
average run length characteristics associated with CS techniques for pro- 
cess shifts of less than about 30 from control. 


Comparisons of extended amounts of data on CS charts before and af- 
ter application of V-mask procedures has dramatically reflected the re- 
sulting improvement in detection power that is indicated in Figure 4. 


In some cases these CS techniques have actually been set up in the 
process control room. The operator responsible for the system actually 
plots information and makes decisions based on the use of a V-mask. Such 
applications have been very successful, since the CS chart itself supplies 
additional information to the operator. He can now visually review past 
data from the chart and develop a better feel for the process system. In 
this way his interest in the control plan is stimulated. This additional 
information comes from the fact that the CS charts have a considerable 
amount of visual estimation power built into them. 


A _“Look-and-See" Approach 


Charting techniques will now be discussed from the more general view- 
point that such charts can be used not only with control procedures, but 
also estimation procedures. Recall the cumulative sum was defined as 


n 
Sn = = (xy - k) 
i=1 


During a period when the process is operating at some level, Uy> it is 
noted that from the original assumptions 


xij (uj, o2) and 
n 
E (Sy) (xij - k)=n (uj - k) = nA;- 


Therefore, when the true process level is different from control, the CS 
points should tend to form a slope of approximately A;, the true differ- 
ence from control. Now n points along the horizontal time axis will cre- 
ate a displacement on the vertical CS axis of approximately nd, whereas 
on a regular chart the vertical displacement is only of magnitude A. It 
is obvious that the scale factor of the vertical axis must be compressed 
if a large number of points are to remain on the chart. A scale factor 
of approximately 2 to 3 sigma per observation has been found to give sat- 
isfactory results, permitting most CS data to remain on the chart. Such 
a scale factor has also been found to supply additional information. 
Basically, this information results from the fact that the scale factor 
compresses most of the relative system noise between points into a zone 
of plus or minus one unit on the vertical axis yet does not seriously af- 
fect the magnification factor na. 


The result of this scale factor is shown in Figure 5. Random normal 
deviates? with distribution N (0,4) were sampled from tables. The first 
points have an input of +lo, the second 20 an input of 0, and the third 
20 an input of -20. These inputs are analogous to process levels of an 
industrial system. Figure 5 represents a typical picture that was exhib- 
ited when 25 such sets of data were plotted on CS charts. The regular 
control chart, with 3 sigma limits centered about zero, is shown first. 
It obviously supplies a minimum of information and does not readily lend 
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itself to estimation of process shifts. 


Now consider the CS chart. Due to the proper selection of scale 
factor the cumulative sum is restricted to a reasonable area - yet the 
three process levels now become distinct. The process noise has been 
largely masked, resulting in a visual tracking of the true system level. 
When the process is operating above control the points form a positive 
slope, when on control the points form a nearly horizontal pattern, and 
when below control they create a negative slope illusion 


Note that in this example one can observe the same type of visual 
impact the CS charts created for the operator on the system when he was 
using the CS chart as a control procedure. Therefore, the CS chart can 
also be used with a "look-and-see" approach to general process study or 
control. 


Assume the data shown in Figure 5 represented process data and that 
neither magnitudes nor time locations of process shifts were known. It 
is now apparent that estimates of both process properties can be made 
from the CS chart. This example indicates that time location estimates 
of process shifts are quite precise when data is CS plotted. Indeed the 
20 sets of random deviates show that the CS chart has excellent detec- 
tion , wer relative to point of process shift. 


The magnitude of the process level relative to the standard, k, can 
be estimated by evaluating the cumulative sums between two estimated 
shift points, n and n-r. This estimate of A (n,r) can be calculated from 


A(n,r) = Sn - Sn-r 
r 


Since the CS chart can show good estimation of the point of process chan- 
ges, the estimates of A should be relatively unbiased. 


Figure 6 shows an example of actual process data. The data is plot- 
ted on both regular and CS charts. Whereas the regular chart only sug- 
gests the occurrence of several process shifts, the CS chart clearly in- 
dicates the existence of a number of process shifts. Such charts can be 
used to advantage in studying past data on process systems. 


Conclusions 


It has been shown that the CS techniques offer the possibility of 
not only improved control but also graphical estimation procedures. This 
supplies the quality control engineer in any industry with valuable new 
techniques. Wholesale replacement of regular charting techniques is not 
proposed. _Rather, selected application of CS charting techniques in con- 
junction with current methods should result in process improvement. CS 
techniques are new and further sampling studies and mathematical treat- 
ment is necessary to allow their full exploitation. 


Actual CS plotting of your own data is a simple way for one to dev- 
elop an intuitive feel for the possible benefits. If this article re- 
sults in the reader's taking this initial step, then its purpose has been 
fulfilled. 
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Table I 
Zone Score (y) 
= 3 
/7 7 ] 
-2 
1 -2 -2 
2 -2 -4 
3 -2 -6 
4 3 -3 
5 -2 -5 
6 -2 -7 
7 -2 ~9 
8 1 -8 
9 -2 -10 
10 1 -9 
1l -2 -11 
12 -2 -13 
13 -2 -15 
14 1 -14 
15 1 -13 
Table II 


CONTROL MASK EXAMPLE 


D= 5.2 D = 5.00 


“Cut-and-Try" Mathematical 
M=1.9 M = 1.75 


| ; 

| ll 

| 
i 


112 


1) 


2 


3) 


4) 


5 


~ 


6) 


7) 


8) 


9) 


1961 ASQC CONVENTION TRANSACTIONS 


Anonymous (1960). Cumulative Sum Chart - New Quality Control 
Tool. Chemical and Engineering News, March 21, 50-51 and cover. 


Barnard, G. A. (1959). Control Charts and Stochastic Processes. 
J. R. Stat. Soc. B, 21, 239-271. 


Dixon and Massey (1951). Introduction to Statistical Analysis, 
New York, McGraw-Hill. - 


Ewan and Kemp (1960). Sampling Inspection of Continuous Pro- 
cesses with No Autocorrelation Between Successive Results, 
Biometrika, 47, 363-380. 


Goldsmith and Whitfield (1961). Average Run Lengths in 
Cumulative Chart Quality Control Schemes. Technometrics, 
February. 


Page, E. S. (1954). Continuous Inspection Schemes, Biometrika, 
14, 100-115. 


Page, E. S. (1355). Control Charts with Warning Lines, Bio- 
metrika, 52, 243-257. 


Page, E. S. (1957). On Problems in Which a Change in Parameter 
Occurs at an Unknown Point, Biometrika, 44, 248-252. 


Shewhart, W. A. (1931). Economic Control of Quality of Manu- 
factured Product. New York; D. Van Nostrand. 


| 
4 
= 


VARIATIONS FLOW ANALYSIS FOR PROCESS IMPROVEMENT 


Norbert L. Enrick* 
University of Virginia 
Graduate School of Business Administration 
Charlottesville, Virginia 


Variations Flow Analysis is a technique of evaluating the transfer of variations 
in stock, when the product from several machines at one processing stage is fed 
randomly to the several machines of the succeeding stage. This paper describes pro- 
cedures, based on modifications of range methods for analysis of variance, which have 
been found of value in a large number of applications. The methods are illustrated 
with examples from yarn manufacture, but parallel applications in chemical processing 
and other industries are apparent. 


INTRODUCTION 


Variations Flow Analysis represents a special adaptation of analysis of variance 
technique, for tracing the transfer of variations in stock from process to process in 
multi-machine and multi-stage processing. By comparing actual variations against ex- 
pected values in each department, it is possible to isolate and correct any places 
where excessive variations are induced into the processing sequence. Corrective 
action can then be concentrated on the places so isolated. 


The analysis procedures have been simplified, so that the average supervisor in 
the industrial plant will understand them sufficiently to make his own applications. 
This feature of the procedures has been verified in actual practice and checked in in- 
dustrial training courses over the past five years, in which attendants representing 
over a thousand industrial supervisors were given sample problems to work out. 


The techniques described here are now used routinely in over a hundred plants, 
representing spinners and weavers of synthetic and natural fiber textiles. However, 
there are many other industrial processing organizations, particularly in the foundry 
and in the chemical plant, where these techniques of Variations Flow Analysis should 
be applicable in a similar manner. Thus, even though the discussion in the following 
may refer to within-machine and between-machine variation in a spinning process, 
parallel application in other industries, such as to variations within and between 
lots in Bessemer steel rolling become apparent. In describing the latter type of ap- 
plication, Weaver (14) deplores the difficulty of explaining variance analysis by 
means of conventional sums-of-squares techniques, a problem which the procedures in 
this paper seek to avoid by the use of methods involving ranges. 


The general model for which the Variations Flow Analysis techniques described here 
were developed is as follows: Production, from raw material to end product, involves 
several processing stages. In each stage, the product from one set of machines feeds 
at random into the several machines of the succeeding processing stage. At any stage, 
a blending may occur, whereby several units of the product are combined. A typical 
production process of this nature is manufacture of yarn in a spinning mill. The pro- 
duct from one set of machines feeds more or less at random into the machines of the 
next processing stage. Production is of the semi-continous p...essing type. Strands 
of product are extruded on a continuous basis on various spindles of one process; and 
when a package of several pounds has been produced, it is removed and then fed into 
the next machine. By means of successive attenuation of the stock, a rather bulky 
sheet of stock coming from the picker machines is gradually reduced to a fine thread 
of yarn in spinning. 


*Based on work done while Associate Director and Head of Operations Research, 
Institute of Textile Techiology, Charlottesville, Virginia. 
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At any processing stage, a special type of blending or "doubling" may occur, 
wherein several strands are combined into a single new strand. Usually, there is at 
least one such blending, which occurs in the drawing process. Here several strands 
are fed into the drafting rolls, where they are blended together. At the same time, 
attenuation by drafting takes place, which is usually large enough so that at any 
one output position of the frame, the individual strand of blended sliver produced 
has approximately the same weight per yard as each of the several slivers fed. It is 
apparent that the physical operations of blending and drafting are equivalent to the 
arithmetical process of adding and then averaging. Therefore, one of the results ob- 
tained is a reduction in variations of stock in accordance with the well known stand- 
ard error of the mean effect. This statistical effect may have been recognized in 
intuitive form by Sir Richard Arkwright, when he developed the drawing frame (8). 


In a typical mill, there may be from two to ten pickers, feeding more or less 
at random into from a hundred to three hundred cards. Each of these machines has 
one feed position and one output position. There may next be ten to thirty drawing 
frames, each with four input and output positions, followed by ten to thirty roving 
frames, and from fifty to three hundred spinning frames. A roving frame may have 
from eighty to 120 spindles, and a spinning frame may have from two hundred to four 
hundred spindles. The differences in number of units in each process is accounted 
by differences in production rates employed. 


Depending upon the degree of diversification of a mill, there may be various 
lines of flow, with a particular raw material going over a particular set of machines 
to proauce a particular yarn style. The Variations Flow Analysis must, of course, 
be carried out separately for each line of flow. 


SUITABLE MEASURE OF VARIABILITY 


In deciding upon a suitable measure of variability, consiveration must first 
be given to the basic unit of measure. This unit, as obtaine: from tests known as 
sizing tests, is the stock weight or more specifically the linear density, usually 
expressed in such terms as “grains-per-yard" for sliver, and in the rather complex 
term “number of 840-yard lengths per pound" for roving and yarn. In listing the 
weight values for various yarns alongside the standard deviations, and repeating this 
for rovings and slivers, the author has observed that in general the standard devia- 
tion increases in direct proportion to increases in linear density. Thus, it appears 
that the coefficient of variation is a good measure of variability, since it permits 
ready comparison of different yarns, rovings or slivers, without need to refer to the 
particular weight involved. In the analysis shown here, the coefficient of variation, 
v, in per cent is used; however, it is apparent that the same analysis could be car- 
ried out in parallel manner in terms of standard deviations. 


In tracing the flow of variations through successive processing stages, the fol- 
lowing variation coefficients, in per cent, have been found to be of importance in 
each processing department: 


1, Within-machine variation coefficient, v,,. 

2. Between-machine variation coefficient, vp. 

3. Departmental-overall variation coefficient, vo, which in view of the general- 

ly normal patterns, of v, and v, is the Pythagorean total: 
v2 + vé q@) 

When a machine has several output positions or deliveries, such as a drawing 
frame, v, may again be broken up into within-delivery and between-delivery components. 
However, where this has been done by the author, it was usually found that the 
within-delivery component was of relatively minor magnitude. The analysis can 
therefore be simplified considerably by ignoring these smaller components. 


Usual sizing test practice in a mill is to weigh S5-yard lengths of sliver, 12- 
yard lengths of roving and 120-yard lengths of yarn. Since the mill usually has a 
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large amount of such sizing test data available, taken for the primary purpose of con- 
trolling average stock weights in pr ing, it app 8 logical to use these same 
weighings also for analysis of variations. However, variance length curve analysis 
of textile strands, has shown thit the observed variation along a strand varies, 

being largest for small lengths and decreasing asymptotically as the lengths increase 
(13). Fortunately, for the range of weights normally encountered in textile pro- 
cessing, the conventional test lengths used in a mill, fall within the asymptotic 
area of the variance length curve. Thus, the use of existing mill data in Variations 
Flow Analysis appears justified. The practical conventions were, cf course, developed 
long before the statistical methods of variance length analysis, and related auto- 
correlation and correlogram techniques had arrived. 


The case of sampling lengths, just as the prior case of blending, shows how the 
practical man can apply statistical principles correctly to mill problems, even 
though the theory of statistics as such has not as yet provided means to analyze such 
problems. There is, however, another field in which the practical man, and the 
practical mill production supervisor in particular, seems to have been less suc- 
cessful. This is in the field of realizing the effects of machine to machine differ- 
ences. In particular, the supervisor may have been exceedingly good at setting up 
each of the machines in each department to produce good quality, with good uniformity. 
However, between machines, there may exist certain small but cumulatively important 
differences in roll diameters, gearings, settings end tensions, contributing to dif- 
ferences in weight between slivers, rovings and yarns. In the final assembly op- 
eration, weaving, these differences will then affect fabric appearance quality. 
Variations Flow Analysis will usually bring out these differences, as well as their 
likely causes, thus spurring corrective action. 


FLOW OF VARIATIONS 


From the general nature of textile processing as described, and the types of 
variation in weight found to be of importance, certain basic patterns governing the 
flow of variations from process to process become apparent. These patterns may be 
stated in the form of four Flow Rules, which have been numbered, so that distinct 
reference can be made to each in subsequent discussions in this paper. The Rules are: 


Flow Rule 1: Since generally several machines at one process feed more or less at 
random into the machines of the subsequent process, the departmental overall variation 


of the fi rocess is input source of n-machine variation o nex 
process. 

Flow Rule 2: n=! vari a ula d 
sult of the i rom the prior process and the effects o additional varia 
that may have been introduced by the machines themselves. 

Flow Rule 3: To the extent to which differences in average level between machines 
exist, they thereby contribute to departmental overall variation. The larger this 


between-machine variation, which is superimposed on within-machine variation, the 
greater will be the departmental-overall variation. 


Flow Rule 4: Where channelling (or non-random hg K of stock occurs, the departmental 
overall variation may not show up immediatel e subsequent within-machine vari- 
ation. Instead, it will then appear in the a departmental overall vari- 
ation of the process. For example, if there are differences in the average level be- 
tween two groups of carding machines on the same stock, but each group channels into 
a distinct group of drawing frames, then this between-group difference on the cards 
will not show up in the within-machine variation in drawing; but it will show up as 
between-machine and therefore departmental overall variation in drawing. This fourth 
Flow Rule may be considered a special case of non-random flow, not covered by the 
first Flow Rule. 


It is evident that from the configuration of variations data in a mill, one may 
predict the existence of channelling without ever having set foot in the plant. This, 
if fact, has been done successfully in numerous cases. It should be noted here that 
channelling, as described above, is always detrimental where the process into which 
the stock is channelled includes blending or doublings. In simple terms, if the some- 
what heavy stock from one group of machines is not blended with the somewhat lighter 
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stock from the other group, then the “evening out" of differences cannot take place. 
In more technical terms, channelling of stock, by its selectivity or non-randomness, 
interfers with the (previously discussed) standard error of the mean effect in re- 
ducing variations. 


Important improvements in variation, and greatly enhanced overall benefits, have 
accrued to mills that have recognized and eliminated channelling and its detrimental 
effects. This is especially important since differences in weight or linear density 
of stock may represent corresponding differences in such factors as degree of paral- 
lellization of the fibers, effectiveness of combing and carding action on the stock, 
and efficiency of removal of non-spinnable waste, such as short fibers, leaf and 
trash. 


Allowing for Blending 


It has already been demonstrated that the process of blending or doublings, ac- 
companied by compensating draft, is equivalent to totaling and then averaging. Ac- 
cordingly, by recourse to the standard error of the mean, and the assurance of the 
Central Limit Theorem (15) in case of (usually slight) deviations from normality, we 
may write the expectation formula: 


Ywe = Vop/nt (2) 


where v,. represents the within-machine variation expected at a given process, v 
represents the departmental overall variation of the prior process, and n equals 
number of strands combined by blending in the given process. 


This formula for the effect of blending of doublings has been found to work well 
in practice. However, a small allowance is usually added to the value of v,,. to take 
account of the effects of drafting during blending. 


All aft 


Where drafting occurs without blending, the expected effect of draft might be 
expected, from analogy with (2) above, to be written: 


Vwe = (op) X (draft) * (3) 


Where draft refers to the linear density of the stock fed divided by the linear 
density of the stock produced. Unfortunately, this formula does not work out well in 
practice. The reason is that, unlike the case of blendings where various strands of 
stock are combined at random, drafting without doubling is a non-random process along 
linear lengths cf a single strand. The lengths exhibit generally a high degree of 
auto-correlation end non-normality. Thus, since there is absence of both random samp- 
ling and normality, the Central Limit Theorem becomes inapplicable. In place of 
Equation (3), it has been found necessary to set up special tests to evaluate the ef- 
fects of draft, or to utilize empirical allowances (4,7). 


CALCULATION METHOD 


From the constant need to control stock weight in a mill, abundant test data are 
usually on hand for the analysis of the variations in stock. Table I shows an ab- 
breviated* set of such data, representing randomly selected machines from each of 
which four random bobbins were tested. The following ranges were obtained: 


Note: The 48 test results used in the illustration here represent a relative 
minimum of data. Usually, twice as many or more test results are readily available 
in the mill, and should be used, so as to minimize the loss in precision of the 
range method as against the sums of squares approach. The relative efficiency of the 
range in estimating the standard deviation has been the subject of prior inves- 
tigations (5, 6, 9-11). 
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1. The range within each machine, R,- 


2. The range-overall, representing the range on any particular day across any 
row of test results, R,- 


3. The daily cross-range of the machine averages, R,- 


From Table II, the calculation of v, is clear. The estimate of v, is shown by 
Methods 1 and 2. While Method 1 is statistically the less efficient, it has the 
advantage of being more readily understood by foremen and other nonstatistical per- 
sonnel. In particular, the calculation steps of Method 1 are parallel to the steps 
in determining v,,, and can thus be grasped intuitively; while on the other hand the 
additional steps of Method 2, shown in Line 10 seems to present considerable dif- 
ficulty to intuitive explanation. 


Any excessive between-machine variation could now be found from the squared ratio 
of the average of the cross ranges to the average within-machine range, as shown by 
David (2). However, for purposes of direct comparisons of coefficients of variation 
from process to process, as‘illustrated in Table VI, it has been found desirable to 
use a test involving the ratio of vo to vy directly. This is the modified F-ratio, 
which for the example at hand is found to be: 


Vo/Vy* 5.39/3.97 * 1.36 


From Table IV, this ratio is found to be greater than the minimum Faog ratio 
of 1.133 required for significance at the 5 per cent level, corresponding to re4, 
k=12, thus showing that the overall variation is significantly greater than the with- 
in-machine variation. This indicates the presence of excessive between-machine dif- 
ferences, causing significant differences in average weight of stock coming from dif- 
ferent frames. The causes, be they in roll weighting, tensions, or gearing, need cor- 
rection so as to standardize the effective drafts among all frames. 


More formal variance analysis procedure, using conventional sums of squares 
techniques, would have yielded slightly different estimates of variation coefficients 
and again a significant F-ratio for the excess of departmental overall over wirhin- 
machine variation, as shown by the data in Table V. 


A TYPICAL CASE HISTORY 


The typical case history, in Table VI, may be used to illustrate how the method 
of Variations Flow Analysis works. For each processing stage shown, 20 samples were 
tested, with each sample consisting of four specimens per machine.’ Accordingly, it is 
found from Table IV that any F = 1.097 indicates a significant excess, at the 5 per 
cent level, of overall variation over within-machine variation. 


For the first process, carding, the vg of 5.2 is significantly higher than the 
Vw of 2.8, the Feratio being greater than 1.097. Based on Flow Rule 3, this means 
that there are excessive differences in average level between the machines in the 
carding process. A subsequent investigation indicated that off-standard trumpet sizes 
and gearing were responsible for these differences. 


The next process is drawing, in which a blending or doubling of sixteen strands 
of sliver takes place. Since generally several cards feed a particular drawing frame, 
the voof carding may be used to calculate the expected within-machine Variation in 
drawing, based on Equation (2). Thus: 


vwe 5.2 + 4 16 


Table VI shows that the actual within-machine variation in drawing is only 0.7 in 
terms of per cent variation coefficient, which is considerably below the 1.3 theo- 
retically expected from the formula. Accordingly, one must suspect that the normal 
pattern of flow, usually expected from Flow Rule, 1 may have been upset by channel- 
ling, resulting in the pattern observed under the special case when Flow Rule 4 
applies. A subsequent check revealed these suspicions to be true. There were 
actually two makes of cards, Saco-Lowell and Platt, which were located in two sections, 
A and B, of the card room. The Saco-Lowell cards in section A had a slightly higher 
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Table I 
Test Results on Spinning Bobbins. Data in Terms of Yarn Number* 


Range, Range of 
Machine Number 43 27 5 16 Overall Averages 
55.5 51.8 56.0 52.0 4.2 
Week 1 50.2 54.2 $7.1 56.1 6.9 
54.8 50.1 55.2 54.1 5.1 
52.1 54.1 51.1 53.0 3.0 
Total 212.6 210.2 219.4 215.2 
Average 53.2 52.6 54.8 53.8 2.2 
Range Within 5.3 4.1 6.0 4.1 
Machine Number 8 39 7 12 
50.3 59.9 55.5 58.8 9.6 
Week 2 58.8 58.2 50.2 56.1 8.6 
54.0 56.3 51.6 59.2 7.6 
53.8 57.0 54.7 57.1 3.3 
Total 216.9 231.4 212.0 231.2 
Average 54.2 57.8 53.0 57.8 4.8 
Range Within 8.5 3.6 $.3 3.1 
Machine Number 21 29 41 32 
50.4 55.4 58.8 58.6 8.4 
Week 3 52.1 56.3 57.3 59.2 7.1 
51.6 56.1 56.8 55.7 5.2 
53.5 58.1 53.9 57.1 4.6 
; Total 207.6 225.9 226.8 230.6 
Average 51.9 56.5 56.7 57.6 5.7 
Range Within 3.1 2.7 4.9 3.5 
. Grand Total: 2639.8 Grand Average: 55.0 


~ 


*Yarn Number represents the number of 840 yard lengths of yarn per pound 
weight. 
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Between 
Within- Departmental Overall Machine- 
utation S Method 1 Method 2 
a. Total of Ranges in Table I 54.2 73.6 12.7 
b. Number of Ranges Used, g 12 12 3 
c. Average Range, R = a/b, 4.51 6.13 4.23 
d. Tests per Range, r or n, r= 4 4 ng=4 
e. Conversion Factor from 
Table III 0.483 0.483 0.472 
f. Standard Deviation, 
T=zacxe 2.18 2.96 2.00 
g. o Estimated by e+ at 
of 2 
h. Variation Coefficient, %, 
= 100 o / Grand Average, 
(Grand Average =< 55.0) 3.97 5.39 5.0* 3.63%* 
aA a a 
i. Symbol for v Vy Yo. Yoo Vv, 


* is found from vet vile 2 
1 
=f(3.97)7+ 3.63)" - (3.97)7/4 = 5.0 
** v, is used only as a stepping stone for finding Yoo" 


Subscripts: wwithin-machine, o = departmental overall, a = between machine 
averages, b= between machines, net. Carets, “ indicate that the values shown are 
determined from sampling estimates. 
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Example: Given Average Range=4.52, and 12 Ranges (g=12) with each Range based 
on a sample size of 4 items or specimens (n= 4), then the Conversion Factor is 0.483, 
and the Standard Deviation is 4.52 x 0.483 = 2.18. 


Source: Reciprocals of values of c, obtained from an extension of H. A. David's 
table in Biometrika 37, 271-280 (1950). 


* If, in place of Average Range, R, the Average Range Per Cent, UR, is used, then 
the Factors yield the Coefficient of Variation. 


Number of Sample Size ( Number of Tests per Range), r or n 
2 0.781 0.552 0.465 0.417 0.385 0.344 0.319 0.303 0.285 
3 -813 -565 +472 -420 346 -322 +304 286 
4 -826 -571 -422 389 -305 287 
5 -840 -575 -467 -323 -305 287 
6 -847 -578 -478 -391 +324 287 
7 -578 -478 -426 +324 306 287 
8 -855 -581 -481 +426 +392 -350 287 
9 -862 -581 -481 427 -392 +350 - 306 287 
10 862 -581 -481 -427 -392 +324 306 287 
14 -870 -585 -483 394 -351 +325 -306 .287 
15 -870 483 +427 +325 307 287 
13 -870 -585 -483 +394 -351 -325 287 
20 -877 -588 -483 +325 .307 288 
25 -877 588 -483 -351 -325 -307 -288 
30 -877 - 588 -483 -395 -351 -325 -307 288 
40 887 -486 -430 -351 -307 
| 
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Table IV: VALUES FOR TESTING SIGNIFICANCE OF DIFFERENCE BETWEEN 
VARIATION COEFFICIENTS, IN VARIATIONS 
FLOW ANALYSIS (MODIFIED F-RATIOS, 
Fmod.» 95% CONFIDENCE LEVEL) 


Number of Sample Size (Number of Tests per Sample), r a 
10 1.453 1.223 1.150 1.113 1.091 1.066 1.052 
12 1.393 1.197 1.133 1.101 1.081 1.059 1.046 
14 1.351 1.178 1.121 1.092 1.074 1.053 1.042 
15 1.33% 1.170 1.115 1.088 1.071 1.051 1.040 
18 1.293 1.151 1.103 1.078 1.063 1.046 1.036 
20 1.272 1.141 1.097 1.074 1.060 1.043 1.03%4 
25 1.234 1.123 1.084 1.064 1.052 1.038 1.030 
30 1.208 1.110 1.076 1.058 1.047 1.0% 1.027 
40 1.173 1.093 1.064 1.049 1.040 1.029 1.023 
50 1.145 1.081 1.056 1.043 1.035 1.026 1.020 


Example: Given 12 samples (k>12) each having a sample size of 4 specimens 

(r = 4), then Fuod, 18 1.113. If the actual ratio of the Variation Coefficients, 
departmental-overall to within-machines exceeds this 1.113, then the departmental- 
overall variation may be considered significantly greater than the within-machine 
variation, indicating lack of standardization of effective drafts among the 
machines. 

Source: From formula Food. = 1+ (F-l)/r, where F is given by Merrington and 
Thompson, Biometrika 33, 80 (1946), with adjustments for use of ranges, based 

on an extension of data tabulated by David, Biometrika 37, 271 (1950). 
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(2) 
Computation Between Between Machines Experimental 
Weeks Within Weeks Error Total 
a. Enter Squared 857.42 212.62 55.55 
Totals + 891.5, +210.2, + 51.8, 
+ 890.9 +219.4 + 56.0 
etc. for all etc. for all 
12 totals 48 totals 
b. Sum of "a" 2,323,610 281,562 145,548 
c. Tests/each "a" 16 4 1 
d. b= c 145,226 145,390 145,548 
e. Sum of dj - dg- d - & 2 d,-C 
Squares = 48 om we 
f. Degrees Freedom 3-1=2 3 4-1) = 9 36 48-1= 47 
g- Mean Square, e— £ 24.0 18.2 4.4 
h. Feratio 81:8 
i. Significance, % Nil 99.5 
8, 82 82 83 83 J1,2,3 
j. Component of cl 
Variance = 0.36 = 3.88 4.4 8.21 
k. Std. Dev., 45" 0.60 1.86 2.10 2.87 
1. Variation Coeff. 1.09 3.38 3.82 5.09 


* C # Correction Factor (Grand Total)”/ Total Tests» (2,639.8)2/ 
48 = 145,178 


= 
122 
Sources of Variation 
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AR ONS A 


Variation Coefficient, Per Cent 


Within-Machines Departmental Overall 


Modified F-Ratio 


Processing Stage ed F 

or Department (v,) Vo) (Vo/V,,) 
Carding 2.8 5.2 1.857* 
Drawing 0.7 1.8 2.571* 
Roving 2.0 2.8 1.400* 
Spinning 3.2 3.4 1.063 


k = 20 is 1.097. 


* Significant at 5 per cent or better, since Fuod’ for ra# 4, 


PROCESS VARIATION (S COMPOSED OF TWO MAJOR PARTS - 
WITHIN MACHINE VARIATION ANO BETWEEN MACHINE VARUTION 


AVERAGE LEVEL 


CURVE REPRESENTING 
PATTERN OF vARIATION 


THE DIAGRAM TO THE LEFT, 
EACH MACHINE (S$ SHOWN TO Have THE 
SAME PATTERN OF VARIATION (WITHIN 
‘ ’ MACHINE VARIATION) 

/ avehace / 


: HOWEVER, DUE TO DIFFERENCES 
BETWEEN MACHINE vamaTion—/ THE MACHINES, THE OVERALL 
VARIATION IS INCREAT CO. 


Figure 1 -- How within-machine plus between-machine variation 
combine to produce overall variation. (Reproduction from 

Enrick, N. Le, "Quality Control," courtesy Industrial Press, 
New York.) 
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draft, thus delivering somewhat lighter stock than the Platt cards in section B. Cor- 
respondingly, the Platt cards with a somewhat lower draft, were delivering heavier 
stock. This difference in stock weight had contributed to the high departmental over- 
all variation on the cards, which was noted above. Now, when pushing the cans of 
sliver from the cards to drawing, the tender would naturally push them the shortest 
distance. And, while there was a certain amount of randomization of the cans, in 
general the relatively light stock from the Saco-Lowell cards would go to the group 
of drawing frames closest to section A, while the somewhat heavier stock from the 
Platt cards would go to the group of drawing frames closest to section B. Within each 
drawing frame, the variations input was thus either from Saco-Lowell cards or from 
Platt cards. Following Flow Rule 2, each drawing frame would thus reflect the group- 
overall variation fed fron either section A or section B of the cards, which was less 
than the departmental overall variation, for the simple reason that the group-overall 
for each section did not include the effect of difference in average level between 
the two groups. Thus the full amount of departmenta) overall variation was not fei 
to any one drawing frame, resulting in an actual within-machine variation which was 
less than would have been expected under fully random flow of stock to each drawing 
frame. 


One might suspect from this example that channelling is actually beneficial, 
since the variation observed within the machines was less than expected under random 
flow. Actually, however, only a temporary advantage is observed. The differences 
between the two card groups continue to be reflected in the form of differences in 
weight of the stock, and will now show up in a high overall variation in drawing. 
Examination of Table VI shows that the actual overall variation is indeed high, with 
a coefficient of 1.8 per cent. The channelled flow has prevented the lighter stock 
from the Saco-Lowell cards to meet the heavier stock from the Plett cards in the 
blending operation of the drawing frames. This has minimized the effectiveness of 
blending (in particular, the effectiveness of the standard error of the mean law to 
operate) and has resulted in a higher overall variation than would have been obtained 
under fully random flow of stock. In particular, without channelling it might have 
been expected that the within-machine variation in drawing would have been only 
slightly above the 1.3 per cent predicted from Flow Rule 1 and Equation (2). The 
overall variation, in the absence of differences between drawing frames, would have 
been at a corresponding level. The actual overall variation, at 1.8 per cent, must 
thus be considered unduly high as a result of the channelled flow. It is interesting 
to note that the mill subsequently shifted the carding machines, alternating the two 
makes among adjoining floor positions, so that in the future there would be relatively 
automatic randomization of the flow from the two types of cards the drawing frames. 
This insured better blending in drawing and, combined with a correction of off-stand- 
ard trumpets and gearing on several of the cards, resulted in substantial improvements 
in card and drawing sliver uniformity of weight. 


Continuing the analysis, it will be noted that in roving the within-machine 
variation exhibits a coefficient of 2.0 per cent, which is generally in accordance 
with expectation, based on Flow Rule 2, and a small allowance for the effect of draft- 
ing in roving. The overall variation of 2.8 per cent is significantly in excess of 
the within-machine coefficient. Applying Flow Rule 3, a check of equipment was made, 
which revealed differences in gearing, roll diameters and cone belt starting positions 
among the roving frames, to be responsible for this excess variation. 


In the final yarn making process, spinning, the within-machine variation with a 
coefficient of 3.2 per cent is consistent with the variations input from roving, de- 
partmental overall, under Flow Rule 2. The overall variation in spinning is not 
significantly higher than the within-machine variation, indicating that there are no 
harmful between-machine differences between the spinning frames. 


The Variations Flow Analysis thus demonstrated places in carding, drawing and 
roving where excessive variations were being induced into the processing. Moreover, 
from an examination of the Flow Rules, the nature of this excessive variation is 
known. In this particular case history, the subsequent corrective actions taken, as 
outlined above, resulted in an eventual reduction in the overall variation coef- 
ficient of the yarn from a relatively medium 3.4 per cent to a relatively good 2.6 
per cent. This is only one example, however. The validity of Variations Flow 
Analysis has now been confirmed in experience in over a hundred plants. The impor- 
tance of the technique is enhanced by the fact that, accompanying reductions in 
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variation of linear density or stock weight, there are improved processing conditions, 
in the form of lowered strand breakage rates in spinning the yarn and in subsequent 
weaving, storage yarn, and improved appearance quality of the more uniform product 
obtained. 


DERIVATION OF MODIFIED F-TABLES 


The Modified F ratio values have been computed from Merrington and Thompson's 
tables of the distribution of Snedecor's F, using the expression: 


Where F is an abbreviation for F(k,r). In calculating F,,, the use of ranges 
was considered, by making an appropriate reduction of approximately ten per cent ir 
the Degrees of Freedom, based on Duncan's tabulated data (2). Further details of 


derivation were previously published (3,4). 
ACKNOWLEDGEMENT 


Appreciation is expressed to Dr. L. H. Hance, President and Dr. Jack Compton, 
Vice President and Director of Research, of the Institute of Textile Technology for 
guidance and encouragement in this work. Computer programming, including checks and 
supplementary calculations, were performed by Charles F. Comer and Sally A. Duren in 
the Operations Research Division of the Institute. 


BIBLIOGRAPHY 


1. David, H. A., (1950), "Further applications of range to the analysis of variance." 
Biometrika, Vol. 37, pp. 271-280. 


2. Duncan, A. J., (1955), "The use of the range in comparing variabilities." 
Ind. Qual. Control, Vol. 11, Nos. 5,8. 


3. Emrick, N. L. (1960) “Quality Control" 4. ed., The Industrial Press, New York 13, 
N. Y. 224 p. 


4. Enrick, N. L. (1960) “Variations Flow Analysis" Technometrics, Vol. 2, No. 3 
pp 373-386. 


5. Grubbs, F. E. and Weaver, C. L., (1947) "The best unbiased estimate of population 
standard deviation based on group ranges," J. Am. Stat. Assoc., Vol. 42, 
pp. 224-41. 


6. Hartley, H. 0. (1950) “The use of range in analysis of variance," Biometrika, 
Vol. 37, pp. 271-280. 


7. LeDuc, H. A. (1950) "Statistical quality control in the yarn mill." Paper pre- 
sented at meeting, Am. Soc. for Qual. Control, Metropolitan New York Section, 
April 29. 


8. Leigh, Evan (1873) "The science of modern cotton spinning," 2nd Ed., Palmer & Howe, 
London, England. 


9. Patnaik, P. B., (1950), "The use of mean range as an estimator of variance in 
statistical tests." Biometrika, Vol. 37, pp. 78-87. 


10. Pearson, E. S., (1932), "The percentage limits for the distribution of range in 
samples from a normal population (n up to 100)" Biometrika, Vol. 24, pp. 207-17. 


11. Pearson, E. S. and Hartley, H. 0., (1954), "Biometrika tables for statisticians." 
Cambridge Univ. Press. 


12 


Tippett, L. H. C. (1925) "On the extreme individuals and the range of samples taken 
from a normal population." Biometrika, Vol. 17, pp. 364-387. 


| 
i 


1961 ASQC CONVENTION TRANSACTIONS 


Townsend, M. W. H. and Cox, D. R. (1951) "The analysis of yarn irregularity." 
J. Textile Institute, Vol. 42, (Proceedings), -pp. 107-113. 


Weaver, Wade R., (1948), “The Foreman's View of Quality Control," Ind. Qual. 
Control, Vol. 4, No. ll. 


Wilks, S. S., (1943), "Mathematical statistics." Princeton University Press, 
Princeton, New Jersey. pp. 81-82. 


A 
126 
| 15. 
? 
. 
N 
. 


A PRACTICAL ANALYSIS OF A SUCCESSFUL SQC INSTALLATION IN 
HIGH QUALITY, HIGH STYLE GARMENT MANUFACTURING 


S. Peter Spalding, Quality Control Supervisor 
C. F. Hathaway Company, Waterville, Maine 


At the risk of sounding like an inquiring reporter, I am going to ask those of you 
who do not already have an SQC program several questions. Are you willing to sever con- 
nections with the traditional, and embark on a new, profitable and proven method of con- 
trolling your quality? Are you satisfied with the quality level of your product, or do 
you wish to steal a march on competition and become a leader in your field? Are you 
capable of approaching seemingly radical ideas with an open mind, and are you capable of 
recognizing and admitting past managerial weaknesses or deficiencies? 


If in answering these questions it becomes apparent to you that there is room and 
desire for improvement in your organization, then perhaps this paper can be of benefit 
to youe For it tells, in some detail, of the successful installation of an SQC program 
at the Hathaway Shirt Company's largest plant, located in Waterville, Maine. Perhaps 
some of the ideas and methods I shall tell you about will be advantageous and adaptable 
to your situations. I sincerely hope so, for in this day of sometime shoddy American 
production it behooves all of us to leave no stone unturned in searching for tools and 
methods which will provide us with a maximum level of desirable quality. For those of 
you already having SQC installations, I hope this text will be interesting and that it 
will promote an interchange of ideas which would’ be beneficial to all of us. 


Hathaway has been a name synonymous with quality for over a hundred years. We 
proudly consider ourselves the style and quality leader of the shirt manufacturers in 
this countiy. It would be safe to say that quality is Hathaway*s life blood, and, 
should it become diluted or tainted, Hathaway would undoubtedly fail. Our intention for 
the future, as it has been in the past, is to expend all possible effort and thought to 
achieve a maximum quality level unequaled in the American market. 


It was with this intent in mind that Hathaway's management asked Kurt Salmon 
Associates, Inc. of Washington, our consulting engineers, about the feasability of in- 
stalling an SQC program. Mr. Robert E. Heiland, who incidentally is Chairman of the 
Textile and Needle Trades Division of the ASQC, explained the potential benefits to be 
derived from such a program and shortly thereafter, in the fall of 1958, the first steps 
were taken toward the installation of SQ. 


Hathaway had the traditional type of 100% inspection found in most garment plants 
today. This inspection system, however, was probably unique in the industry in that 
there were no fewer than eleven 100% inspection points located in the dress shirt stitch- 
ing area. This same type of intensive quality inspection was also being applied to 
sport shirt and Lady Hathaway production. Although the protection received from this 
type of costly and detailed inspection procedure provided a garment of superior quality, 
we did not feel fully satisfied that we were utilizing our efforts to the best and most 
efficient advantage. The following points seemed to be increasingly apparent. 


1. 100% inspection is monotonous with the inspectors becoming robotlike in 
their reactions. Such mechanical inspection is not sensitive to subtle but 
undesirable changes in quality. 


2. Traditional inspection systems are designed to sort good work from bad 
but are incapable of preventing the occurrence of bad work by catching it at 
the needle as it occurs. To illustrate this fact let us consider an operator 
on collar closing, who might be producing bad quality due to machine troubles 
or emotional disturbances. Her defective work would remain undetected until 
all subsequent operations on the collar were performed and the defects reached 
inspection. Often a period of severa] days elapsed before detection occurred. 
By that time, many dozen bundles had been produced in a defective manner by 
the collar closer. What we wanted was a method of catching bad work as it 
occurred at the needle, and thus prevent defective work from accumulating. 


3. Another weakness of 100% inspection was that unit supervision was respon- 
sible for its functioning. Inspection, therefore, became far more susceptible 
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to production pressures than common sense said it should be. Furthermore, 
supervision was not equipped with the proper facilities, personnel, or time 
to maintain a continuous check of each operator's quality level. 


4. Various quality experiments were conducted at Hathaway over the years 
prior to 1958, in efforts to assure better quality. At various times a 
staff member would become "Quality Director" and take over the supervision 
of 100% inspection. All these experiments failed, sometimes because line 
supervision thereupon attempted to abdicate its responsibility, and also 
from a lack of statistical knowledge, and the resulting inability of staff 
and production people to evaluate fairly most quality problems. 


In proposing a new quality installation for Hathaway, we were assured that the 
above mentioned weaknesses of 100% inspection systems would not occur with the plan of 
SC known as "Skip Bundle Sampling". The following seven claims were made for the pro- 

) 


gran, 


1. That it would be capable of application without adding to the total manu- 
facturing cost. 


2. That it would be simple enough to be administered by existing plant per- 
sonnel. 


3. That it would be sufficiently discriminating to apply pressure immediately 
to those operations where quality had deteriorated below a tolerable level. 


4. That it would be sufficiently powerful to force real improvements at 
trouble spots when the trouble occurred. 


5. That it would provide accurate statistical data to enable management to 
really control quality and would provide a reliable, objective grading sys- 
tem for each operator in the plant. 


6. That it would be flexible in order to permit shifting of control empha- 
sis as the need appeared. 


7. That it would recognize the basic fact that sewing room quality is con- 
trolled almost entirely by people, not machines. 


After a careful survey study of our quality problems, costs, and desired quality 
levels, the use of Skip Bundle Sampling appeared to be feasible and economical to our 
management. I was assigned to spearhead and to supervise the installation under the 
technical guidance of Kurt Salmon Associates, Inc. 


It was decided to begin our installation in the Dress Shirt Stitching room, rather 
than in Cutting or Finishing. This was done for several r e It d that the 
greatest quality improvement potential lay in the stitching room. Furthermore, by main- 
taining the status quo in the finishing room, we obtained an objective barvmeter of 
quality changes taking place in stitching. 


At this point I shall briefly describe the Skip Sampling plan we ha’’e adapted for 
our stitching operations. Bear in mind this plan applies only to Stitching, not Cutting 
or Finishing in which other sampling plans are used. Furthermore, one of the attractive 
features of Skip Bundle Sampling is that it is a flexible quality tool and, therefore, 
while the plan I shall describe applies to most stitching operations, we have changed it 
on certain occasions to better fit specific needs. In general terms, here is the way 
our plan works. 


We knew, as I am sure you do, that some operations contribute much more heavily to 
the overall defect level than do others. In our Dress Shirt Stitching, where some 
seventy to eighty operations are pérformed on each shirt, there were ten or twelve op- 
erations accounting for over 80% of the in-process defects. Thus, in each unit, and by 
unit is meant a group of related operations which manufacture a particular part of the 
garment, there were one or two critical quality operations. It was at these operations 
we first directed our attention. In most instances I did the initial week or two of 
sampling or inspection. Then qualified personnel were trained and assigned to these key 
operations as "samplers". 


Samples are drawn, on a random basis, from bundles representing one-fourth of each 
critical operator's daily production. It is important to note that this is a random 
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selection and does not involve every fourth bundle. Bundles used for sampling purposes 
are drawn from the most recently completed production. This gives an evaluation of work 
as it leaves the operator's needle. From each bundle selected, twenty items are 
screened. These items are also selected on a random basis from within the bundle. The 
effect of such random selection is, of course, to insure that the operator will not be 
able to predetermine what portion of her work or which items will be screened. If upon 
inspection these items satisfy our established quality requirements, the bundle is re- 
leased to the next operation. If a defect (or defects) are found in the bundle, it is 
returned to the operator responsible. 


It is at this point that the true effectiveness of Sampling becomes apparent. Under 
our 100% inspection system defective work was removed from a bundle, tied in a separate 
stack by the inspector, and returned to the operator by unit supervision. The operator 
then repaired her mistakes and returned them to the inspector who replaced them in. the 
correct location in the original bundle. With Skip Sampling the procedure is markedly 
different. After a random sampling of twenty pieces has been completed and the exact 
results recorded on appropriate forms, 2 bundle with one or more defects (that is, a 
rejected bundle) is returned to the operator by the sampler. Defective pieces are not 
pulled out of the bundle, however, and the responsible operator is required to reinspect, 
on her own time, the entire bundle, repairing all defective pieces she may have sewn. 
She then returns the repaired bundle to the sampler who resamples it following the orig- 
inal sampling procedure of choosing twenty random pieces. If acceptable the second time 
the bundle is released. if not acceptable it is returned, after notation has been made 
on the operator's record card, and the operator must rescreen the entire bundle again. 
Work returned a second time is returned through unit supervision. Obviously, rein- 
spection of entire bundles on the part of a piece worker is costly and distasteful to 
her. If an operator is not quality conscious at the inception of sampling she quickly 
becomes that way as her earnings reflect her carelessness. 


I should like now to digress for a moment and make the following observation, 
stated first by Mr. Heiland, and later corroborated by our experience. Defective work 
is seldom produced on a consistent basis. If an operation produces 4% bad work it is 
more than likely producing 2% defective work 90% of the time and 22% defective work 10% 
of the time. In short, defective production seems to be of a cyclical nature. 


With this fact in mind, let us return to our explanation of Hathawayts Skip Bundle 
Sampling. When an operator produces defective work it puts us on warning that perhaps 
a defective cycle is beginning. To counter this condition the skip interval of one 
bundle in four being selected ceases and every bundle produced by the operator under 
suspicion is sampled until two consecutive bundles are found to be acceptable. One 
effect of this feature is to ensure that the cause of defective work is corrected, 
whether it be machine trouble, operator carelessness, or a training or methods problem. 
When the quality level deteriorates badly, the amount of sampling required increases 
until it becomes impossible for the sampler to cope with it. At this point, management 
must do something to return conditions to normal. 


Thus, in Skip Bundle Sampling a variable proportion of production is ned, not 
a fixed proportion. The extent of variability depends on the amount of defective work 
being produced; the higher the percent defective, the larger the proportion of produc- 
tion which must be inspected. 


Since this sampling method appeared sensitive to immediate fluctuations of quality, 
as they occurred at the needle, and since it seemed to provide sufficient penalty to 
discourage sloppy work habits, it was hoped that shortly after sampling installations 
were made, 100% inspection operations could be eliminated. Such proved to be the case 
as we progressed through the "parts" units. 


Using conventional notations for the sample size, n; the average bundle size, N; 
the acceptable number, c; the skip interval, Ss and the clearance interval, m; our plan 
may be expressed as follows. 


N = 21 c=0 m=2 
n = 20 S=4 


At this point, it should be noted that the highest average outgoing quality level, 
cr AQQ, occurs when a process is running slightly under 6% defective. At this point the 
AQQ level is 2.73% defective. The AOQQ decreases as the process runs higher than 6% due 
to the increased amount of inspection required. This relationship is shown by Figure l. 
This AOQ factor compares most favorably to our old 100% inspection protection. I under- 
stand that recent studies made by various garment engineering firms indicate that 100% 
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TABLES OF VALUES OF AVERAGE OUTGOING QUALITY AND AVERAGE 
FRACTION INSPECTED FOR THE SAMPLING PLANS: 
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inspections actually average only slightly higher than 50% effective. 


Before actual steps were taken to install Sampling, management, floor supervision, 
and engineering got together to establish definite, realistic, and mutually acceptable 
quality standards. Opinions existing at that time were far from unanimous and the re- 
sulting "meeting of the minds" in itself helped to improve our quality situation. 


In further anticipation of our SQC installations, detailed records were kept at our 
100% inspection stations. These records were to give us an invaluable means of compari- 
son between the effectiveness of traditional inspection vs. Sampling. We shall consider 
several of these records shortly. 


I shall not attempt to describe in detail the actual installation of Skip Sampling 
in each individual unit. It will suffice here to make the following observations which 
apply to the overall progress of the Stitching room program. 


Although there was 100% skepticism on the part of supervision, management, and 
operators when this program was originally suggested, it has now become fully accepted 
and supported by practically all concerned. This welcome fact has been accomplished in 
several ways. 


At no time was the installation pushed ahead without proper preparations having 
been made. These consisted of a thorough evaluation of existing conditions and objec- 
tives and a detailed and candid explanation of Skip Sampling methods to all persons con- 
cerned. The program was "sold" to the operators largely on the basis that Sampling 
would improve the quality of work coming to them and thus make their jobs easier. 
Furthermore, the operators were assured tha a detailed record of individual earnings 
would be maintained and, if it appeared that Sampling was unfairly reducing their earn- 
ings, temporary bonuses would be applied to piece rates. This was done in several in- 
stances. 


It is significant to note here that average unit earnings have increased noticeably 
since the installation of Sampling, even after the temporary bonuses had been removed. 


This has been brought about not only by the improvement in quality, but also by the fact 
that sampling was quick to pinpoint areas of mechanical, supervisory and training weak- 
nesses. It is an accepted fact now at Hathaway that the ability of Sampling to focus 
attention on weaknesses found in all phases of production has been nearly as beneficial 
and desirable as its ability to control and improve an individual operator's quality 
output. 


Although individual operators whose quality was considerably below our acceptable 
level were severely affected by Skip Sampling, our program has been installed with vir- 
tually no Union opposition. This is, in my opinion, strong testimony to the value of 
advancing slowly and patiently, with an open mind and sympathetic approach to individual 
problems as they arise. 


One of the most important elements involved in the construction of a quality pro- 
gram is the selection of capable people to become inspectors, or samplers as we call 
them. We have tried to select people who have the capacity to grow with their job and 
who have the ability to win the confidence of the unit people they sample. With only 
one or two exceptions our 100% inspectors were incapable of becoming samplers. In sev- 
eral cases we took a top producer from an operation and made her a sampler. This 
apparent robbing of production paid for itself in a relatively short time, as the 
sampler's ability not only to detect bad work but to demonstrate corrective and effi- 
cient methods quickly helped raise unit production levels. Regardless of previous ex- 
perience, we have stressed that our samplers must be fully familiar with all quality 
specifications, that they must approach their stitchers with humility and a sincere 
desire to assist, that they must be unbiased, and that they must be capable of making 
routine decisions on their own without becoming dependent on floor supervision for 
advice or comfort. We have been fortunate in our selection of samplers, as is attested 
by the low turnover of samplers, the measurable improvement in quality, and the lack of 
Union opposition. 


A most valuable tool provided management by an SQC program is found in the form of 
statistically accurate and reliable records. We have tried to keep our recordings con- 
fined to the most essential details. Should a need for more detailed knowledge of 
specific aspects of quality production be desirable, additional information can be 
amassed with a minimum of effort and cost by using existing data-collection methods. 
Indeed this has been done several times in the past. Our current utilization of records 
is as follows. Once a week production management has a quality meeting, chaired by our 


| 
é 
# 
| 


132 1961 ASQC CONVENTION TRANSACTIONS 


production vice-president. Prior to this meeting a weekly grading sheet is published 
which lists all Stitching, Cutting and Finishing units sampled, their production, number 
of samples taken, and their percent defective quality. Units are graded as "good" if 
their defect level is less than 1%, "fair" if they average 1% to 28%, and "poor" for an 
average of over 24%. Another sheet is run off every week which lists all operators 
averaging over 4% defective per week. These reports are discussed in detail at the 
weekly meeting and corrective measures are taken when necessary. Operators who appear 
on the defect list for the first time are verbally warned by supervision. Repeat 
appearances within a three month period result in written warnings and possible dis- 
missal. 


We can see by now that Skip Bundle Sampling directs management's attention to those 
operations, people, machines, and materials which are responsible for poor quality. It 
does this in several dramatic and convincing ways. One is by providing sound statisti- 
cal evaluations of unit and individual quality levels. Another, as previously explained, 
occurs when an operator or operation runs consistently worse than an acceptable level. 
When the volume of required inspection is too much for the sampler to manage, super- 
vision has to take some action to straighten the problem out and get work flowing again. 


So far I have briefly described the Skip Bundle Sampling plan used in our stitch- 
ing rooms. We have also installed Sampling in our cutting and finishing rooms so that 
now we cover all production operations in both Dress and Sport shirt lines at our 
Waterville plant. We are currently installing Sampling in our staple stock plant at 
Lowell, Massachusetts. 


In our installations in Cutting and Finishing, different types of sampling plans 
were used. These differ mainly in the size and type of sample taken. The penalties 
invoked for poor quality are much the same in all cases, with the operator being re- 
quired to reinspect his entire production unit in which a defect occurs. For the sake 
of time and space I shall not go into the details of the various sampling plans used 
but, should anyone be interested, I would be happy to discuss them at another time. 


Now to consider the all-important question of "How well has Sampling worked for 
Hathaway?" I can definitely say it has worked very well. To offer specific evidence 
of quality improvement, I have included in this paper Figures 2 and 3. Figure 2 illus- 
trates the percent defective work being found at different parts units before and after 
Sampling. It also reflects the reduction in personnel needed to perform the quality 
inspection. I believe this is a rather impressive comparison of the past and the 
present. Figure 3 indicates the percent of defective work reaching our final inspectors 
from all stitching operations before and after Sampling. Again these figures speak for 
themselves. Similar improvement has been statistically verified in Cutting and Finish- 
ing. The seven specific claims made earlier in this paper for Skip Bundle Sampling 
have been corroborated by our experience. We are confident that our quality has im- 
proved and will continue to do so. I do not wish to lead you into thinking that 
Hathaway has solved all her quality problems. Such is not the case. Currently we are 
struggling to apply foolproof controls to our fabric flaw problem. But we feel that 
Sampling has given us the tools to work with and, with perseverance and ingenuity we 
should soon be even farther ahead of our quality position of five years ago. 


In closing I should like to bring forth these observations. 


1. As should be expected, this type of program creates more problems than it 
solves during the initial stages of installation, but it solves many more than 
it creates after becoming established. This is a fact which is all too easy 
to lose sight of during the period of "growing pains". 


2. Skip Bundle Sampling is not a program to be embarked on lightly. It is not 
capable of being established effectively if it is applied in haste or without 
competent engineering advice. We found that generous initial expenditures of 
time and money, where they appeared necessary, were most profitable. Hathaway, 
as is the case with most garment manufacturers, had no statistically trained 
staff capable of initiating such a program. Without the knowledge provided 

by outside professional engineers, this program would never have risen off the 
ground. 


3. Finally, as is the case with all aspects of production, this type program 
is ultimately dependent on the desire of management and supervision to make 
it work. Without cooperation from all production departments, such an instal- 
lation will bog down and fail. My function in this program has been largely 
one of sales and diplomacy. It is imperative that someone unite such tradi- 
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QUALITY LEVEL COMPARISON OF DEFECTS FOUND AT DRESS SHIRT PARTS 
MANUFACTURING--PRESENT VS. SURVEY PERIOD 


SURVEY PERIOD POST SAMPLING PERIOD 
11/24-12/12/58 1/14-2/11/61 
NUMBER OF BER OF NUMBER 0 % 
DEFECTIVE |INSPECTORG PEFECTIVE |SAMP. PTS). SAMPLERB CHANGE 
CUFFS 2.10 3 1.44 1 1 -31.4 
FRONTS 7.19 4 3.56% 2 2 -50.4 
ors 
or 
COLLARS 14.66 4 7.49% 3 -48.9 
e 
BACKS 4.69 2 1.93 1 7 -58.9 
TOTAL 28.64 13 14.42 7 6 -49.65 


THIS PERCENT IS TOTAL OF NUMBER OF SAMPLING POINTS 


FIGURE 2 
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AND AFTER ELIMINATION OF 100% INSPECTION IN MANUFACTURING UNITS. 
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COMPARISON OF REPAIRS FOUND AT FINAL TRIM AND INSPECT BEFORE 


2. 4. 
— Verecre || Xderectwe derectwe DEFECTWE 
1. | CUFFS 0.24 0.45 0.59 0.13 
2. | SLBEVES 0.03 0.08 0.02 0.08 
3. | BACKS 0.12 0.14 “0.01 0.15 
4. | FRONTS 0.32 0.30 0.36 0.47 
5. | COLLARS 0.78 0.10 0.25 0.31 
TOTAL-PARTS UNITS 1.49 1.07 1.23 1.14 123.5 
1. | JOINING 
COLLAR SET 
2. | AND FINISH 3.10 0.83 2.09 2.20 
BUTTON AND 
3, | BUTTON HOLE BAND) 
SLEEVING AND 
4. | FELLING 4.79 0.47 0.89 1.27 
CUFF SET 
AND FINISH 
. 6.33 1.29 1.91 1.00 
TOTAL-ASSEMBLY UNITS || 14,022 2.52 4.89 4.47 -68.6% 
ALL MANUFACTURING || 15,71 3.59 6.12 5.61 64.3 
FLAWS, HOLES, BTC.|| 1 96 2.08 1.79 2.13 + 7.9% 
GRAND TOTAL 17.67 5.67 7.91 7.74 56.2% 


“INDICATES % CHANGE BETWEEN COLUMNS #1 AND #4. 
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tionally independent and uncooperative departments as Cutting, Stitching and 
Finishing. Once these groups start pulling together and become "sold" on 
the capabilities of SQC there are few problems that can not be solved quickly 
and easily. 


If you have an earnest and eager desire to improve your quality position, and are 
capable of channeling your management's efforts toward this goal, you will find, as 
Hathaway has, that Skip Bundle Sampling will be a most effective tool. 


(1.) "Skip Bundle Sampling--A New and Economical Method for Process Control in the 
Sewing Room."" Robert E. Heiland, Research Director. Kurt Salmon Associates, 


Ince; Textile Quality Control Papers, Vol. 6, 1959, published by The Textile 
and Needle Trades Division, ASQC. 
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MOISTURE CONTROL IN THE CASHMERE INDUSTRY 


by Stanley Breen, General Manager 
Forte' Engineering Company, Norwood, Mass. 


INTRODUCTION 


During the past four years a new system of measuring and controlling 
the moisture content of fibrous materials - protein - cellulose, and syn- 
thetic has been in use by a number of manufacturers in many differert 
branches of the textile industry. This paper will review how this system 
is applied in the processing of Cashmere. 


Cashmere is the fiber from a goat native to certain areas of the 
Middle and Far East. In the treatment of this fiber it is necessary to 
maintain moisture control during wet and dry processing operations and 
finally bring out the finished product at a standard moisture content. 


Moisture content of Cashmere, in common with all textile fibers, is 
significant in two main areas of operation: processing control and account- 
ing: It is well know that weighing, conveying, carding, spinning, knitting, 
dyeing, weaving, and finishing operations are only controllable if the 
amount of fiber is known and the moisture adjusted for optimum quality and 
efficiency. The accounting operation is concerned with the value of the 
textile fiber from the standpoint of price. In buying and selling, value 
is dependent on moisture content because almost all textile fibers are 
hygroscopic. Fibers vary in normal moisture content from 0.5% to 13%, and 
in price from 10¢ to $20. per pound. Cashmere is currently priced in the 
range from $5. to $12. per pound. Contracts are written for payment based 
on clean dry fiber present to a specific regain, or shipped weight with a 
moisture content not to exceed a specified amount. In either case it is 
necessary to measure and report moisture to the most accurate value 
possible. 


The system described determines the moisture content to the most 
accurate standards now possible, which permits delivery of the best 
possible product at a cost within controlled and predetermined schedules. 


THE MEASURING SYSTEM 


The fundamental approach of the moisture control system described here- 
in is the following: An instrument capable of measuring large samples quick- 
ly is used on the production floor. This instrument is calibrated by labora- 
tory methods and monitored by these same laboratory methods. We thus su>sti- 
tute rapid secondary measurements for slow primary (laboratory) methods and 
use the laboratory method on a reduced scale to "Quality Control” the 
secondary method. The secondary system was designed to measure a large 
sample to achieve better representation of a lot. The combination of the 
large sample and rapid measurement insures representation of the lot and 
“present time" information which permits accurate control during processing. 


Moisture content measurements during production were made on instru- 
ments which measure the dielectric constant of the fiber. Examples of in- 
struments are shown in Figures 1, 2, 3, and 4. Figs. 1 and 2 showa 
Forte' Electronic Moisture Content Analyzer, Model K. Figs. 3 and 4 show 
the Model D. The Model K determines the moisture content of a 1+ pound 
sample of bulk or staple fiber. The Model D measures the moisture con- 
tent of an entire ball of wool top, which weighs between 16 and 22 pounds. 


A measurement is made by inserting the sample in a test cell, clos- 
ing a lid, and reading a meter which is scaled in arbitrary units of 0 to 
100, proportional to dielectric constant. An electric field passes through 
the entire sample producing a dielectric effect essentially equal to the 
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average dielectric constant of the mass within the field enclosure. By 
choosing proper field constants, the change in dielectric constant 
measured depends mainly on water rather than extraneous impurities or con- 
ductivity. To calibrate the instrumert, samples are drawn from production, 
weighed to 20 ounces for the Model K or weighed as "catch weight" on the 
Model D, and "read" on the panel meter. Parts of these samples are then 
sent to the laboratory to be oven dried by conventional means. A plot is 
made of meter reading vs. laboratory moisture and smooth curves drawn for 
lines of constant weight. Sub-samples are drawn for laboratory tests, 
rather than drying entire samples, for economy of time and practibility. 


PRACTICAL USE IN CASHMERE PROCESSING 


An actual case will be used to explain and illustrate the system. 
In the processing of Cashmere the terminal moisture can be adjusted by 
controls available to the machine operator. Thus, it is only necessary 
that he have access to instantaneous, accurate, and representative informa- 
tion regarding the moisture content of the product. Machine operators draw 
1+ pound samples and control the moisture content by sampling, adjusting, 
resampling, and readjusting. Experienced operators have little difficulty 
reaching control and there is little or no oscillation or mmting. In 
addition, they mst record hourly, the moisture content of a sample as 
produced. A typical hourly control diagram from production at Forte'- 
Fairbairn, Inc. is shown in Fig. 5. The measurements were made on a 
Model K by shift machine operators. 


Calibration of the instrument was effected by drawing five sub-samples 
of 10 grams each from the 20 ounce sample. These were dried to constant 
weight at 105°C and the loss of weight reported as moisture. The average 
of the five was used as a data point and plotted against "meter reading”. 
Monitoring of the instrument is accomplished by a similar process except 
samples, in triplicate, are drawn only once per twenty-four hour working 
day. The moisture content by Forte' Model K is tabulated against moisture 
content by laboratory; differences are taken, and a control diagram plotted. 
A control diagram covering a typical 8 day period is shown in Fig. 6. This 
control diagram contains the differences due to sampling and oven drying 
only 50 grams from the 570 gram total sample. 


A further check is made independently by the Quality Control Depart- 
ment by means of opening bags of Cashmere on the loading dock, drawing 
samples, and measuring them for moisture by laboratory methods. The 
sample selection is purely random. These tests are expensive to make be- 
cause drawing the samples requires cutting open bags of stock; hence they 
are made infrequently. This final control measures what the customer 
receives. A plot showing a typical four month period is shown on Fig. 7. 
The moisture content at this point is dependent on storage conditions as 
well as the initial moisture condition of the stock. 


SUMMARY 


Our experience indicates a machine operator can measure and control 
the moisture content of in process Cashmere to a specification value with 
a standard deviation of o = 0.25% moisture content. The instrument corre- 
lation to laboratory, including the limited sub-sampling, is approximately 
the same order of magnitude. Long term studies of shipments indicate 
stock is delivered at slightly lower than specification moisture content. 
Without modern instruments and systems of this type, it was not possible 
to control moisture content of processed Cashmere. 
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SQC TRAINING FOR THE CHEMICAL INDUSTRY 


R. S. Bingham, Jr., Manager, Quality Control Department 
Coated Abrasives Division, The Carborundum Company 
Niagara Falls, New York 


Abstract 

S paper discusses what chemists and chemical engineers need to know about 
statistical quality control and other statistical methods, where they should seek these 
ideas, what types of training should be of interest tothem, kinds of problem orienta- 
tion they are apt to have, the pros and cons of undergraduate and postgraduate training, 
the possibilities of on-the-job training, what's being done by interested societies and 
what's missing. With such tremendous scope it should be evident that these are simply 
notes to stimlate discussion. 


WHY TRAIN? 


College interviewers are asking potential employees, "Have you studied statistical 
quality control or statistics?" Properly used, the payoff from these methods is sub- 
stantial in time, new knowledge, superior process control and reduced losses, "why 
train?" is better restated in terms of "who, how soon, how much, where, and what 
topics?" In spite of the best theoretical approaches, much profitable chemical process- 
ing is still based on empirical understanding and in some cases is nothing less than a 
highly prized art. Effective process control speeds identification of key process var- 
iables, and interrelations. Recent work has gone far to link empirical studies, their 
results and interpretations with theoretically postulated mechanisms. Statistical 
methods play a large part in this attack by engineers, armed with computers, experiment 
designs, process simulation and systems analysis on intricate multi-million dollar 
plants. Much of this is beyond the scope of this discussion but serves well for refer- 
ence. 


It is as serious to overtrain as undertrain. Training is expensive, takes the 
time of the most knowledgeable people and generally disrupts routine operations. 
Therefore, any plan for an individual or a company must be tailored to its specific 
needs, abilities, attitudes, tempo and degree of sophistication. The returns are 
greatest for those having the least statistical knowledge. This is generally true un- 
til the most advanced methods are used when another higher dollar return level can be 
sought. 


To avoid giving the impression that chemical industry statistical requirements are 
unique, it is useful to describe them in more detail. 


WHAT DO CHEMISTS AND CHEMICAL ENGINEERS NEED TO KNOW ABOUT SQC? 


The statistical quality control needs of chemists and chemical engineers are not 
substantially different from those of quality control people in other industries. Pri- 
marily, they need to have a keen awareness of variation. In addition to this and its 
ramifications, it is important that they associate the differences between their indus- 
try and others with different ways they should use statistical quality control tech- 
niques. For example, they will want to associate systematic and random errors with 
those Parts of laboratory equipment and laboratory methods from which they are likely 
to occur. 7) They should be keenly interested in the concepts of accuracy and preci- 
sion as well as recognizing that laboratory pestine is as much of a process and worthy 
of control as their manufacturing processes. Thus a need for statistical control of 
a laboratory chemical test method will be as important as statistical control for in- 
process assurance. Their interest in sampling will egegnd even further than that of 
piece parts to the general problems of bulk sampling. The general principles are 
the same but the application and interpretation differ. Since chemists and chemical 
engineers are generally indoctrinated in the "how to do it" aspects of laboratory meth- 
ods, it is extremely instructive to select a relatively simple method and perform rou- 
tine tests on a standardized solution to get a feeling for the kinds of variation that 
tests contain. These technical people have been heartily criticized for thinking only 
in terms of a single measurement primarily because they do not recognize the need for 
Suitable replication of the test. 
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WHERE SHOULD THZY GsT THESE IDEAS? 


Ideally, these notions should be planted in college and nurtured in industry. As 
Suggested, a little bit of on-the-job experience helps considerably in understanding 
variability. This means that laboratories, usually a starting place for greenhorn 
chemists or engineers, provide opportunities for simple application of control charts 
and statistical methods. Too often we neglect college laboratories where calibration 
of laboratory equipment and the examination of repetitive test results can give addi- 
tional training. The basic idea of quantitative malysis and the repetition of tests or 
experiments in Physics Laboratories should provide additional strengthening of the con- 
cept of variability. 


Unfortunately ,from a training standpoint,the standard deviation of most chemical 
and physical test methods is relatively small. This suggests that destructive tests of 
some kind should be considered for greater emphasis on the importance of controlling 
test method precision. Practical demonstrations of the strength of materials is suit- 
able. 


In mathematicscourses other than statistics, a study of calculus offers an oppor- 
tunity te review the effect of errors or discrepancies on estimation. In engineering 
course work, studies of instrumentation and systems design should permit some discus- 
sion of the effect of noise or random signals and the difficulties associated with de- 
lay in automatic sampling and testing devices. 


To effectively consider training in the chemical industry it is important to 
appreciate the problem orientation chemists and engineers have on the job. These can 
be divided into three groups: Research and Development, Laboratory Control, and Pro- 
cess Control. The basic knowledge for all of these is substantially the same, with the 
exception that the first two have greater emphasis on more precise experimentation and 
measurement methods. On the other hand, if any substantial investment is involved, a 
new process or a modification of an old one will eventually go either into a pilot 
plant, semi-works plant, or have one or more full scale plant runs of its own for veri- 
fication purposes. This means that production engineers need to know a good bit about 
statistical quality control and statistical methods for planning experiments with full 
scale equipment. Since the field is very large the interested reader is referred to 
reference \1) for additional information from a problem orientation viewpoint. 


PROS AND CONS OF UNDERGRADUATE AND POSTGRADUATE TRAINING 


Ideally, every chemist or chemical engineering graduate should have at least one 
statistics course under his belt. Unfortunately, the concepts of statistics and SQC 
are only one among many that belong among his guiding principles on graduation. This 
competition and the only recent appreciation of the value of statistical methods in the 
chemical industry has allowed the competition to get there first with the result that 
statistics is not included in the curriculum. 


Fortunately, SQC and the concepts of experimental designs can be gained in post- 
graduate courses offered in many universities. Two things are needed on the part of 
the student; one is awareness of the need to know SQC, and the second is sufficient 
sensitivity to appreciate the application of the methods in his own industrial environ- 
ment. 


Unfortunately, postgraduate courses vary widely in content, availability and even- 
ness (reflecting the competence and experience of the instructor). Furthermore, chem- 
ical industry employees are frequently transferred from one geographical region to an- 
other making a consistent plan difficult to complete. The best solution to date for 
the ambitious engineer seems to be that of the 6 week summer session offered jointly by 
North Carolina State College, Virginia Polytechnic Institute, University of Florida and 
Oklahoma State University. Under this program a M.S. in statistics can be obtained in 
about 3 summers. A contribution must be made by both the student and his company to 
make the 6 weeks stay each year possible. 


ON THE JOB TRAINING 


Because of these factors, there is a substantial effort in medium and large size 
companies to provide training of chemists and chemical engineers on the scene. Al- 
though this is a function of the size of the companies and of their overall capability 
in statistics, it is possible through the use of consultants to conduct on-the-job 
short (5-10 day) courses. Most of the companies that have undertaken such programs 
have done so because they are progressive toward new methods and because they have had 


| 
| 
| 
i. 


PHILADELPHIA, PENNSYLVANIA 147 


successful applications of the techniques in one or more plants. In many cases, the 
training has been undertaken for a limited number of students selected eagayee of their 
ability to sense problem requirements and to identify types of solutions. 


To make the most of this kind of training, the chemical companies must provide in- 
ternal consultation for interested students who learn statistics and proceed to try it 
out. Nothing is more deadly than to try statistical methods on a job, assume that you 
have used them correctly, write a glowing report and then discover a basic misunder- 
standing of an assumption has scuttled all your dollar savings. 


In many instances the complexity of the chemical problems makes the solution with 
statistical techniques extremely time consuming unless a computer is available. How- 
ever, computers are becoming more and more common in chemical com _— (This adds a 
new dimension to quality control training - how to use ommend 2) Not only does 
this enhance the solution of these more difficult problems but it also makes it possi- 
ble to provide routine solutions for control chart data and things of this nature. 
(Chemical companies are well known for the small size of their technical staff. This 
means that routine data analysis has to pass quite an ordeal of fire before it gets on 
any computer, ) 


Finally, a very important item for any training is strong management understanding 
and support of the activity as well as tolerance of the time that must be set aside for 
continued consultation and follow through. 


SHORT COURSES 


In addition to the on-the-job training courses which run the gamut from basic con- 
trol charts through tests of significance, linear regressions and design of experiments, 
a number of short courses are being offered jointly sponsored by universities and tech- 
nical societies. The Chemical Division of the American Society for Quality Control has 
been prominent in devising and offering new courses of this type. Stemming from an 
original 10 day basic Quality Control course for the Chemical Industry institutec at 
Rochester Institute of Technology, whose content for the most part is identical to 
other basic QC courses but whose overtones and applications are very specifically aimed 
at chemists and chemical engineers, extension has been offered to design of experiment 
10 day courses. In addition to this, based on the publications of Box and Wilson, and 
Box, Hunter, Hader and others, 3 day courses in response surface methodology have been 
offered and successfully carried out in the U. S. These 3 day sessions provide the in- 
terested chemist or engineer with basic notions of how to solve regression problems 
using the abbreviated Doolittle Method, analysis of variance for linear models, the 
principles of replication, ideas on first order and second order designs, augmenting 
first order designs to make second order designs, concepts of lack of fit and mathe- 
matical fitting of response surfaces, 


For the chemist or engineer who has done considerable work with statistics, the 
Chemical Division of ASQC has developed a 10 day course designed to present and discuss 
the latest techniques of statistics that should have good strong application in the 
chemical industry. The first of these was offered at The Harvard Business School in 
1958 and the second was held at The University of Western Ontario in 1959. These 10 
day sessions are not for beginners but have been well received by those attending. 


Recognizing these special requirements of control chemists has led to the design 
of a 3 day course in analytical techniques specifically for chemists. A course enti- 
tled "Evaluation and Interpretation of Physical and Chemical Testing Methods" was 
successfully conducted at Philadelphia in 1960 by the Chemical Division ASQC and ASTM 
for this purpose, 


Two day courses in Evolutionary Operations (3), a concept of G. E. P. Box designed 
primarily to obtain information for improving processes from the normal running of the 
process itself, using 2 and 3 factor factorial designs in plant oper-tion, have been 
carried out over various parts of the nation with good acceptance. hese techniques 
have been selected for teaching due to their special pertinence in this industry. Re- 
ports of successful applications have helped. 


The Gordon Research Conference of the AAAS held each summer provides a different 
kind of training. The sessions on statistics in chemical engineering basically offer 
discussion of papers presented either by statisticians or by chemists or engineers. 
With only sufficient number of engineers to keep the balance on the chemical side, the 
meetings aim to provide full review of what's new as well as what's needed. 
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One very important approach to training for the chemical industry which should not 
be overlooked is that of home study. As yet, no correspondence courses are available 
at the level and for the type orientation that chemists and engineers generally have. 
However, a number of professors have been good enough to provide guidance for engineers 
sufficiently motivated and yet remotely located so that it has not been possible for 
them to study at a university. Obviously, the return is directly proportional to the 
amount of effort the student applies, 


WHAT'S MISSING? 


The primary problem with each of these training methods is that of simlating con- 
trol information and applying it in such a way and with sufficient repetition so that 
confidence is quickly obtained. Statistical methods must be used to be learned and 
appreciated. A second problem in the chemical industry relates to the relatively in- 
_—— demands placed upon chemists and engineers for strong mathematical backgrounds. 
Although the concepts of partial differential equations may be discussed in undergard- 
uate programs, the level of sophistication of chemical engineers in general is not math- 
ematically as great as that of electrical engineers. With the advent of systems 
approaches to automation more emphasis on mathematics should provide greater readiness 
Zor statistics and SQC training. One of the major gaps that exists in the application 
of SQC in the chemical industry lies with failure to recognize instrumentation require- 
ments as well as the random nature of measurement error. We have only begun to bridge 
the gap of optimizing process control. This requires understanding of such techniques 
as dynamic and linear programming, lagrangian multipliers, operational calculus, etc. 


ROLE OF THE SUPERVISOR IN TRAINING 


Another interfering element in the application of statistics in the chemical in- 
dustry has been the failure of chemical management to recognize that statistics and 
chemical engineering are each full time jobs. While there have been exceptional chem- 
ical engineers who have been capable of learning and applying statistical methods to a 
sufficient degree to make the most of them, they are relatively few in number. Those 
that have been successful statistically usually gravitate toward statistical consulta- 
tion in chemical companies. The net effect is the loss of a good chemical engineer; 
however, with effective team operation this loss can be quickly rectified. 


The student must not be led to believe that techniques and chemistry are enough 
for industrial problem solving. His supervisor will offer much more support if he sees 
evidence of proper balance between methods and problem identification, diagnosis and 
practicality of solution including an adequate inclusion of the pertinent human rela- 
tions aspects. However, the supervisor must encourage the student to use his new 
skills at every appropriate opportunity. Obviously, the student mst not play the role 
of a hunter with a new gun in pursuit of any game it can kill, but instead, must pick 
and choose among his methods for the correct one for each particular problem. Students 
must try out their new skills soon after learning them for increased confidence and re- 
tention. Those given adequate encouragement by their supervisors have inevitably done 
better than those feeling that it was just another method. Another difficulty in pro- 
viding training of this type relates to how much experience a chemist or engineer must 
have before he can fully appreciate the value of statistics. In some cases, a life 
time is not enough. For others, spacing the lectures or discussion periods over a year 
can be sufficient to provide opportunity for stimulation and student application. In 
other chemical situations greater success is had with continuous training programs, 
such as 5 day courses. Obviously, it is very difficult to free key people for 5 suc- 
cessive days from any operating division. This has led to the obvious compromise of 
one or more days a week over the equivalent number of weeks with homework assignments 
and on-the-job data collection and analysis as requisite. 


CLOSING COMMENTS 


In considering how to set up anc conduct a training program there are several 
principles to keep in mind to insure success. These are more pertinent to organiza- 
tions unaccustomed to formal training programs than to others. 


1. Start small, put the ideas to siork and aim to show savings in time, effort, 
and preferably dollars, 

2. Try out new ideas before including them in the syllabus. 

3. Get and use competent statistical censultation for planning, reference, 
and ideas on effective presentation. 

. Expand as soon as the students seem ready. 

5. Train continually by plan rather than sporadically. Better to space out 
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a sound plan than rush an over=ambitious one. 

6. Use training aids for maximum effectiveness. These include notes, 
charts, models, simulators, and props as well as change of pace, 
a types of presentation (lecture, discussion, demonstration, 
etc.). 

7. Avoid getting everyone into the act. Select both the students and 
the course content. Tailor-make with examples from your business. 

8. Maintain a strong feedback to assure the students and teacher are 
together. 

9. Select the training staff carefully. Not everyone can or should teach, 
but few admit it. Use the experts for consultation and preparation 
of handouts, concise examples, etc, 

10. Keep the discussion as close to actual plant or business situations 
as possible without getting embroiled in non-pertinent arguments. 
Introduce several new statistical ideas each session and review them 
each time for better understanding. Keep a "What Did He Say?" score- 
card in front of the class to list new statistical terms. 

ll. Encourage, support and recognize the students as they learn and use 
their new learnings. Get others to do likewise. 


SQC has much to offer all chemists and chemical engineers -- not just these doing 
quality control work. Since it is results that count, training must be given to make 
the use of the methods simple and automatic. While zeal on the part of the instructor 
will add fire to his lectures, an enthusiastic student is our immediate goal. 
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S.Q.C. METHODS IN 
TELEPHONE TRANSMISSION MAINTENANCE 


J.B. Pringle, Staff Engineer - Quality Analysis 
Bell Telephone Co. of Canada, Montreal, Que. 


INTRODUCTION 


There are two fairly widespread but nevertheless erroneous notions about control 
charts that have impeded their application in some areas. These are: 


(1) That “our product is different” and so control charts are not 
applicable. 


(2) That our process must be more-or-less in control and the product 
rather homogeneous before we can use control charts to advantage. 


Actually, if we can make satisfactory measurements on our product, its nature has 
no bearing on the applicability of control charts; and control charts can be very useful 
in getting a process into control, as well as keeping it there. The material int this 
talk illustrates both these points. 


STATEMENT OF THE PROBLEM 


We have a continent-wide telephone network, the purpose of which is to allow any 
one subscriber to converse with any other, with ease and dispatch. To this end, con- 
nections must be made correctly and rapidly, and the volume and quality of the speech 
transmission must be satisfactory. The volume of speech transmitted on a switched call 
is the characteristic that we are concerned with here. It depends principally upon the 
attenuation of the various trunk circuits employed. This can be readily measured in 
decibels. 


The toll network consists of thousands of individual facilities or trunks, working 
between designated points; for example we have something like 100 trunks operating 
between New York and Montreal, 370 between Montreal and Toronto, 315 between Toronto and 
Hamilton Ont., etc. These individual facilities may be of radically different design. 
some are carried on open-wire, some on cable, some on microwave radio channels. Carrier 
systems, using the frequency spectrum above the voice range, are used extensively. 
Altogether we may have something like twenty or thirty different kinds of systems, but 
any individual circuit may be carried on one type of system, or may be built up of 
different systems in tandem. The lengths of the trunks differ radically as well, from a 
few miles to thousands. There is at least one thing that all these trunks have in 
common, regardless of how they are made up or operate en-route, they all appear as 
simply a pair of wires at the terminating points. 


The switching of calls through this extensive network follows a definite pattern, 
known as the General Toll Switching Plan. A schematic of this is shown in Figure l. 


With Direct Distance Dialing (DDD) which will be universal on the continent about 
1965, our crossbar toll switching equipment works automatically to channel calls ac- 
cording to this plan. 


Toll offices are classified as:- 


1. Tributary example: Paris, ,Ont. 
2. Toll Center Brantford 
3. Primary Center Hamilton 
4. Sectional Center Toronto 

5. Regional Center Montreal 


We have two regional centers in Canada, Montreal and Regina, and perhaps a dozen in 
the U.S.A., such as White Plains, N.Y., Pittsburgh, Chicago, St. Louis, Dallas etc. 


LCS Classification 720: 70: 548 
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REGIONAL REGIONAL 
CENTER CENTER 
Regina Montreal 
SECTIONAL SECTIONAL 
CENTER CEMTER 
Winnipeg Toronto 
PRIMARY PRIMARY 
CENTER CENTER 
Dauphin Hamilton 
TOL TOLL 
CENTER CENTER 
The Pas Brantford 
TRIBUTARY TRIBUTARY 
Flin Flon Paris 
suBS 


Fig. 1 - General Toll Switching Plan 


In general, calls from tributaries to distant tributaries are routed up this line 
to the regional center, which has direct connection to all other regional centers. It 
routes the call to the regional center of the distant tributary, and then down the line 
to the distant tributary. This process is frequently short-circuited to a degree, where 
some of these links are by-passed by “high-usage” groups of facilities. However, under 
conditions where high-usage groups are found busy, the call is routed via the backbone 
routing of the toll switching plan. We may get as many as nine trunks connected in 
tandem on a toll call. 


It is vital to the successful operation of this scheme that the transmission losses 
actually experienced in the individual trunks do not deviate excessively from their 
designed losses. If we design a trunk for 2 db loss, and we find when we measure its 
loss it is anywhere + §& db from this value, we will be in serious trouble when a number 
of such trunks are switched together for a call. 


What we are interested in here is trunk net loss deviation, the difference between 
measured loss and design loss, or:- 


Trunk net loss deviation = (Measured loss) - (Design loss) in decibels. 


These values, for all our facilities, form a distribution, and this distribution, 
in general, can be considered as‘ approximating the familiar normal shape. 


When this matter was first investigated some years ago (perhaps I should say some 
decades ago) it was thought that the spread of this distribution (+ XJ) might be about 
+2db. It was actually found, in many offices, to be nearer + 6 db. This represents 
a@ standard deviation of 2 db. If we have a toll connection through our maximum 9 trunks 
in tandem, it is like selecting nine individual trunks at random from this distribution, 
and adding them up. The standard deviation of the distribution of connections with 9 
trunks would be x 2 = 6 db, and the total spread of this distribution (+ 307) would 
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be + 18 db. 5% of our 9 link calls would be more than + 12 db. from what they should be, 
resulting in excessive loss and can't hear conditions at one extreme of the distri- 
bution, and howling or extreme echo conditions at the other. This is simply intolerable 


for the successful operation of our switching plan. 


Obviously, improved maintenance was called for. Progress has been made to this eni 
over the years, to reduce the amounts of these deviations, and, in general, to reduce 
the spread of the distribution of deviations. 
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Fig. 2 - Stroke Chart 


This has been done by maintaining stroke-chart histograms of trunk loss deviations 
in toll test rooms, calculating the mean and standard deviation of these distributions, 
and taking corrective action (at least in theory) against all cases where the deviation 
was found to exceed + 2 db. An objective, or bogey, has been set up - & standard devi- 
ation of not more than 1 db and mean of not more than 0.25 db. 


In this procedure, using a stroke chart histogram to judge when action should be 
taken to correct a trunk net loss deviation, progress was made up to a point, but in 
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many offices, after years of effort, it seemed impossible to better a standard deviation 
of 1.35 or thereabouts. Some reasons for this were:- 


(a) So many facilities required attention on this basis that proper investigation 
to determine assignable causes was frequently not undertaken, and a compen- 
sating adjustment on terminating equipment was often resorted to. This latter 
just constituted another assignable cause which wou) be evident if the origi- 
nal trouble cleared. 


(b) ‘There was no clear distinction between assignable and chance causes, and no 
real knowledge forthcoming as to just what value of standard deviation was 
within the design capabilities of the system. 


THE APPLICATION OF CONTROL CHARTS 


In this situation, the use of Control Charts seemed like a good bet. A decision 
was taken to run a trial in one office for a period of one year. In order to judge 
effectiveness, it was undertaken that the standard deviation of distribution of loss 
deviations of these trunks should be brought below the bogey of 1 db. by the control 
chart method. This was in the face of over a year's operation by stroke charts prior 
to this, in which 1.35 db. seemed to be about the attainable minimm. 


Bear in mind that in this application, we are dealing with something having in- 
herently more variability than what we are accustomed to expect in a manufactured 
product; we have many different kinds of systems, in many different lengths, and 
sometimes combinations of systems in some individual trunks. By-and-large, though, we 
can classify our toll circuits in roughly four broad groups:- 


1. Voice frequency cable and open wire 
2. Open wire carrier 

3. WN and K carrier 

4,. Microwave. 


On eachc these first three classifications we ran a separate control chart. In 
the trial office we did not happen to have any microwave facilities terminating. 


All of the trunks controlled by the trial office were less than 200 miles in length 
Separate charts and standards would have been maintained on long facilities if any had 
come under the trial. 


The control charts used were not strictly the conventional X-bar and R charts, in 
that @’for the X-bar chart was computed from the overall distribution of deviations, 
rather than from the average range, and, therefore, it included a component of between- 
group variability as well as the within-group variability. The reason for this is that 
we admittedly are not dealing with things quite so “identical” as manufactured products. 
Even if we were dealing only with a specific type of facility, there are differences in 
length, which could involve quite a few "chance" causes. Hence we allow something for 
between-group variability. This may be reduced as assignable causes are found and re- 
moved, and eventually we may be able to compute 7 ‘from R-bar. 


Let us outline the procedure in detail. A routine had been established whereby, 
each quarter, a large sample of the trunks coming under the control of the office was 
examined for net loss deviation, and from this, X andd “calculated, using the stroke 
chart. It was desired to interfere with this routine as little as possible. ‘ine 
procedure used was as follows:- 


1. Three separate X-bar and R charts were maintained, in the following general 
classifications:- 


a) Voice frequency cable and open wire 

bd) Open wire carrier 

c) WN and K carrier 

2. The quarterly sample was divided into rational ‘sub-groups of 4 each, that is, 
each group of 4 would comprise trunks as alike as possible in make-up and 
length, with the same terminal points. 


3. Measurements were made in each direction of transmission, on each sub-group of 
4, and the results recorded on a form. Each form therefore covered the equiva 
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lent of 2 rational sub-groups of 4 each. X-bar and R were calculated for 
these groups. 


TYPE OF FAX TRE 
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Fig. 3 - Form for Recording Data 


All observations for the quarter in each generel classification were recorded 
on a stroke chart, and computed. 


¥ and R charts were plotted, currently with the observations, using the 
following limits:- 


for the X chart: Central line ¥ =0O 
UL =+ 
n 
LCL 


for the R chart: Central line = R for the base period 
UL = DR 
ICL =0 


All out-of-control points were investigated to find and remove the cause of 
the trouble, with as little delay as possible. All trouble found and removed 
was recorded. All sub-groups showing plots within control limits were left 
untouched. 


At the end of each quarter, revised control limits were calculated for the 
following quarter. These were calculated from the variability experienced in 
the quarter ending, with the following conditions observed:- 


(a) The out-of-control points in the quarter were ignored in the calculation 
of the new limits, because they had been corrected. 


(>) If nevertheless, by chance, the new limits were wider than the previous 
limits, the limits were not relaxed ani the previous limits were retained 
for the subsequent quarter. 


RESULTS 


By this procedure, it was possible to bring the standard deviation of the distri- 
bution of trunk net loss deviations down from about 1.35 db. to 0.93 db. within the 
trial period. This was within the bogey of 1.0 db. and hence a matter of some satis- 


fac sion. 


But to me, probably the most satisfaction came about the third quarter of the 


trial, when reviewing the results with the foreman responsible for carrying out the 


trial. 


I asked him to estimate the amount of effort being expended on the trial com- 


pared to the amount expended previously when it seemed impossible to improve ong = 1.5. 
After some thought, he said:- "Even including the time it took us to learn about the 
control chart method, I think we are spending only about 80% of the time we spent 
previously". This is the meaning of the quotation "Better Quality at Lower Cost". 


The essence of the process is that action is taken only when it is highly probable 


that assignable causes are present and can be found. This directs effort into con- 
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structive channels, and reduces effort expended in pursuing minor causes, difficult to 
find, and which do not contribute much to total variability. Only after major con- 
tributing causes are found and removed, is there much to be gained from going after 
minor things. But the Control Chart, by itself, only points out where action can be 
taken with profit. It is the action taken that improves things. If we don't act to 
discover and remove causes, we won't get improvement. 


What are some of the things the charts brought out? Let us look at some very 
abbreviated versions of some of the charts. 


OF FOUR 


AVERAGE Ob 


RANGE Ob 


4th QUARTER 1958 Ist QUARTER 1959 2nd QUARTER 1959 3rd QUARTER 1959 


Fig. 4A - Abbreviated Control Chart - Cable & Open Wire Voice Frequency 
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Fig. 4B - Abbreviated Control Chart - N & K Carrier 
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Whenever an out-of-control point was recorded, an investigation of the facilities 
in the sub-group involved was immediately undertaken. Frequently it was a range outage, 
with only one circuit in the sample of 4 at fault. This could be traced to a specific 
malajustment, faulty tube, or similar cause. Occasionally an average would be out, 
which would indicate a cause common to all four in the sample, such as the gain ad- 
justment of a carrier repeater, common to all four trunks. 


It was even possible to detect some quite general causes from the pattern of the 
chart above, even although no out-of-control point was involved. For example, a regular 
saw-tooth pattern in the X-bar chart on voice frequency facilities between the same two 
points, indicated that there was an error in the level of the one-milliwatt tone supply 
used in the measurements at one of these locations. Im general, the rule is not to 
take action when a control limit is not exceeded, but this pattern indication was so 
obvious, and the consequences of the fault so general, that corrective action was under 
taken in this instance. 


One quite general trouble disclosed by the charts was an ambiguity in our practices 
in connection with a specific type of system, where a lineup tolerance of + 1 db. ina 
certain adjustment was allowed without any compensation elsewhere. Correction of these 
practices will remove this source of variability. 


There is something about these X charts that a quality control man will spot 
immediately, the tendency of the points to fall above the center line, which is taken 
as zero. The averages tend to have a positive value, rather than the expectancy of zero 


This tendency has a strong annual cycle about it, too. Figure 5 shows the annual 


variation in the quarterly averages of the distributions of deviations in N and K 
Carrier. 


N & K CARRIER 
AVERAGE IN EACH QUARTER 


2 
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QUARTERS 


Fig. 5 - N & K Carrier - Variation in Quarterly Averages 


It is not much different from the averages of all types of facilities. We have a 
great deal of information confirming this annual cycle for all kinds of facilities. 


Insofar as N Carrier is concerned, we have found that this annual cycle can be 
accounted for by the residue of temperature variations that are uncompensated for by the 
automatic temperature regulation of the system, hence the further reduction of this 
variability should be a matter of system design, rather than maintenance adjustment. 
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However, we have not yet determined just why we should have a constant positive 
component of the quarterly averages. The most plausible explanation to date seems to 
be that, in linimupa trunk, a man likes to approach the objective from the lower side. 
He is rather reluctant to raise gains, and as a result tends to leave things, on the 
average, just below the specified gain. 


In this application of control charts, to date, we have bettered our bogey with the 
expenditure of about 80% of previous effort. This is quite gratifying, but there is 
little doubt that adequate investigation of these quarterly average figures to get at 
the causes behind them, would yield us considerably better results. 


SUMMARY AND CONCLUSIONS 
I hope this story has brought out the following points with reasonable clarity: 


1. The control chart method is applicable to anything measurable. It doesn't 
matter what type of product we are dealing with. 


2. We don't have to wait to apply control charts until after we have got things 
in order. We can use control charts effectively and economically just to get 
things in order. 


In conclusion, it might be well to emphasize the simplicity of application of the 
method. Given a certain amount of base-period data, the engineer can compute control 
limits and set up average and range charts for any group of facilities. The testboard 
man can select rational sub-groups of four, make measurements, and plot the average 
and range of each sub-group. If these values fall within the control limits, he takes 
no action. If either value falls outside limits he investigates. In most instances he 
will readily find the trouble, and take action to remove it. At the same time this 
action is likely to remove assignable causes from other facilities as well, since quite 
@ number of such causes will be common to a whole group, not necessarily all represented 
in the sample. In a few cases where the testboard man cannot find the specific trouble, 
he should refer the problem to the engineer, rather than resort to a compenseting 
adjustment. As troubles are found and removed, limits can periodically be tightened up 
to a limit of the system capability. Practically all the work is done at the testboard, 
by the testboard man; and the amount done can be adjusted to whatever is deemed to be 
desirable in view of the amount of trouble that is actually being experienced. 
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MEASURING RELAY CONTACT EROSION 


Ernest R. Lowrey, Central Office Maintenance Engineer 
Tllinois Bell Telephone Co., Chicago, Illinois 


Present day telephone systems employ large numbers of relays for switching, controlling 
and signaling purposes. Usage of such devices is, of course, not confined to the telephone 
industry. In the large majority of telephone applications, relay contacts are designed to 
last the entire life of the particular circuit unit involved. In a few situations, however, 
the type of contact load and frequency of operation result in appreciable wear from electri- 
cal erosion, 


For a particular design of relay the extent of erosion will depend on a number of fac- 
tors. The current density and inductive energy of the load circuit are generally predictable 
for a given design application. The number of operations to which a relay is subjected may 
vary from one location to the next. The length of lead between a contact and its load also 
affects contact life, ani this factor is often highly variable. Environment, length of time 
between operations and certein mechanical adjustment conditions also have an effect on the 
erosion rate. Anticipation of all the interacting factors is a very complex problem from a 


Depending on the number of customers served from the particular building, a telephone 
central office may contain from 50,000 to 40,000 relays, each with an average of 5 sets of 
contacts. The relays are components of functional equipment groups with varying operational 
requirements and rates of usage. 


Under normal conditions, contact life up to 50 million operations can be expected of 
these relays. Their durability can usually be increased considerably by provision of con- 
tact protection in the form of a capacitor in series with a resistor to drain away the excess 
‘current. Such protection is used where it is economically justified, or in cases where cir- 


cuit operation failures due to erosion would be particularly onten, © 


About half of the relays in a central office will have usage not to exceed 50,000 opera- 
tions annually. Thus with normal contact loads, they can be expected to have many years’ 
contact life. On the other hand, there are groups of relays required to operate 1 to 5 
million times a year. Also, there are some on which the usage runs to 25 million operations 
annually. Where there is an appreciable erosion rate on any of these relays, a considerable 
maintenance problem results. 


NEED FOR MEASURING EROSION 

In the period just prior to World War II a number of manual central offices were con- 
verted to dial operation using a newly developed switching system. Prior to actual operating 
experience it had not been possible for the designer to anticipate all the points where some 
contact erosion would occur. There were, of course, restrictions during the war years and an 
urgent need for plant expansion in the period following. These factors prevented for some 
time either addition of contact protection or consideration of circuit changes to alleviate 
erosion problems on equipment already in service. 


The problems which did develop were in certain common control circuit equipments. There 
are from 100 to 300 such units in an office. Routine inspections to check for contact wear 
are quite expensive. Correction of individual cases as they are discovered in investigating 
trouble indications is very inefficient. Frequent maintenance operations in such complex 
circuitry is a potential source of man-made troubles. Available maintenance records in the 
offices involved did not show consistent experience for specific relay contacts. 


Consideration of these factors led to the conclusion that variables sampling of a number 
of particular relay contacts in several offices should answer several questions. It was 
hoped that the wear rate could be estimated with sufficient accuracy to allow scheduling 
maintenance rehabilitation. If such schedules could be set up with an optimum balance 
between cost and risks of circuit failure, over-all maintenance operation would benefit 
greatly. If contact life could be determined, the economic justification for adding contact 
protection to existing plant could be evaluated. 
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FIGURE 1- MICROSCOPE IN POSITION FOR USE 


— 
| 


PHILADELPHIA, PENNSYLVANIA 161 


The problem of measuring erosion on contacts of relays in plant by mechanical gauging 
seemed formidable. However, these contacts are accessible for visual examination from the 
front of the equipment. An arrangement for optical measurement of these dimensions, shown 


in Figure 1, was therefore enanine.* This employs an erecting, wide field microscope, 
20X magnification, with eyepiece tube inclined 45°. The microscope has a field of 4.868 ° 
inches and a working distance of 3.5 inches. The Filar Micrometer eyepiece (12.5x), has a 
movable crosshair controlled by an indexed vernier dial. Illumination is from a small ring 
type fluorescent light designed for use in close-up photography. The gear for clamping the 
instrument om equipment cabinets and positioning the microscope in front of particular con- 
tacts was assembled from readily available components. 


A pair of eroded contacts as seen through the microscope is shown in Figure 2. It will 
be noted that these are bar type contacts, with one placed horizontally and the other verti- 
cally on the actuating springs. Each scale division shown represents .020 inches and each 
of the 100 divisions on the crosshair vernier is .0002 inches. The crosshair is here at an 
indeterminate position for clarity of illustration. Dimensions are established by the 
difference between scale and vernier readings when the crosshair is set at the two ends of 
the dimensions being checked. 


It will be noted that depth of erosion in the horizontal contact cannot be measured 
directly. Also, under some conditions of circuit load there is a transfer of metal from 
one contact to its mate. The particular contact which receives the “build-up” is determined 
by the circuit polarity and there will be a corresponding "hole" in the opposite contact. 
The criterion for serviceable contact life is the amount of contact metal remaining on which- 
ever of the pair has the least. Such measurements must also begin at the bottom of any hcles. 


There result three different contact conditions, as diagrammed in Figure 3. The effec- 
tive height of contact metal remaining can thus be determined by measuring three or four 
dimensions as shown, The type of erosion is determined by visual inspection and then only 
the appropriate dimensions are measured and recorded. In each case, measurements must be 
made with the contacts in both closed and open positions. Under two conditions, height of 
the vertical contact can be measured directly. In all other cases, the effective height is 
the difference between two measurements. 


SAMPLING PLAN 
Each telephone call originating in an office must be served by one of the common control 
circuits under discussion. These circuits are arranged in groups of ten. Any particular 
originating call will generally have access to ten such groups on a random distribution 
basis. In the older offices using this switching system, the lowest numbered idle circuit in 
a group of ten is always selected. 


With this arrangement, the total call load is equally divided among the groups of ten 
circuits. Within the group of ten, however, the usage is heavily channeled to the lower 
numbered units. Circuit No. 1, for example, will handle about 25% of the total group load. 
The share of Circuit No. 5 is about 10% and that of Circuit No. 10 is only 0.7%. 


In these offices, knowing the total call volume, it is possible to choose circuits with 
ten different degrees of use, and to estimate the number of operations each has experienced. 
For this stratified sample, five circuits were chosen at random from each of the ten usage 
choices. Particular relay contacts selected on the basis of maintenance experience were then 
measured in the 50 circuits comprising the total sample. Data were identified with the par- 
ticular subsample of five. 


Since later operating experience indicated that uniform circuit usage is more desirable, 
the entire load distribution plan is on a completely random basis in newer offices. In this 
case, all circuits in the office have experienced the same number of operations. The sub- 
samples of five are not #tratified, and were chosen on a random basis. Since the problem was 
essentially in the older equipment, relatively few measurements were made in newer offices. 
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FIGURE 2 - ERODED CONTACTS WITH SCALE 
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ESTIMATING CONTACT LIFE 
purpose it appeared that three hypotheses should be tested: 


for a particular contact, the life is a straight line function of usage. 
the same contact would behave similarly in different offices. 

the data would be sufficiently homogeneous to enable an estimate of con- 
life to be made. 


For the objective in mind, it seemed adequate to fit a regression line to the 
measured values by the method of least squares, and to make a visual comparison with 


the plotted points. 


However, with 50 points per office a scatter diagram of individual 


values would become quite crowded. 


For most of the measurements taken there are five 


contact height values for each of ten usage levels. 
samples, the visual problem is made manageable. 


By plotting means of these sub- 


This also is compatible with the defi- 


nition of the regression line, y = a + bx as the estimated locus of average values of 
the dependent variable. 


The theoretical assumption is that each measurement of contact height is an obser- 
vation on a random variable which is normally distributed with constant variance about 
the mean value a + bx. An estimate of the variance of the observations is expressed by 
the equation 


s2 
y-x 


This can be computed from the data used to obtain the equation of the regression 
line. From this value the theoretical 3-sigma limit of the distribution is obtainable. 
A plot of this limit would show at what point individual contact height measurements 
could be expected to approach zero. 


EXAMPLES 

From several dozen different contact situations measured, three examples illus- 
trating different stages of wear are presented. It will be obvious that all theoreti- 
cal assumptions of normality, constant variance and exact fit of the regression line 
are not met. Considering, however, all of the possible causes of variation inherent in 
the physical condition, it appears the data show behavior consistent with the hypothe- 
ses. From a practical viewpoint, an estimate of contact life can be made which is 
accurate enough for the purpose intended. 
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In Figure 4 there is a case of moderate wear rate. The regression and limit lines were 
computed from 50 samples in a fixed choice office. The mean value is shown for each of the 
ten subsamples. The mean value of 50 measurements in another office with random usage is 
superimposed. The range of 2ach of the sample groups is indicated by a vertical line. Fit 
of the regression line seems reasonably good, but the distributions are generally skewed. 
There was no observation outside the 3-sigma limit. At about 16 million operations there is 
the expectation of individual contacts worn through. 
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FIGURE 5-MEAN CONTACT HEIGHT-RELAY “Q" 


The behavior of a faster wearing contact in two fixed choice offices is shown in Figure 
5. Two interesting situations are illustrated. The subsample with the highest usage in 
office C contains some contacts which were replaced some time ago. The replacement date is 
not available in the maintenance records. On the surmise that this occurred at nine million 
operations, this sample would fit very close to the regression line. At that point, the 
3-sigma limit indicates that individual contacts completed worn through could be expected. 


Since this subsample is from a different population, it was excluded in determining the 
over-all characteristics. The regression line and variance were computed from the remaining 
95 measurements in Offices A and C combined. 


The two highest usage subsamples in Office A have averages which are minus values. Here 
is a case where a contact build-up has worn entirely through the contact metal of its mate 
and penetrated into the actuating spring. Normally these cases would have been replaced some 
time ago. However, circumstances had delayed the rehabilitation in this office. Even in 
this condition, the contacts remained reliable enough that emergency treatment was not neces- 
sary. 


Por clarity of the illustration, ranges of the subsamples are not shown in this Figure. 
The variance, ranges and general pattern of the distributions are comparable to those in the 
previous case. 
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Extreme wear rate is illustrated in Figure 6. Here there are only nine subsamples in 
two offices which have not been subjected to contact replacement. Since wear-through could 
be expected at about four million operations, the highest usage choices have had replace- 
ments three or four times. The maintenance philosophy has been to treat such cases a choice 

t a time as wear-through develops. The averages of these treated choices are therefore 
widely scattered and have been omitted for clarity of the illustration. Visually they 
appear to conform to about five regression lines parallel to the one computed from the 45 
lowest usage measurements. After seeing these results, it was obvious that contact protec- 
tion should be provided for this case. 
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CONCLUSION 
Optical measurements of erosion on relay contacts in service provided data for statis- 
tical analysis. Where particular contacts with various usage levels were sampled, a regres- 
sion line could be fitted to the data. Computation of a 3-sigma lower limit from the sample 
variance allowed an estimate of when individual contacts completely worn through could be 
expected. The data indicated good conformance to a straight line wear rate and compatibility 
of behavior in different locations where the same type of equipment is in use. 
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NEW DEPARTURES FOR OPERATIONS PROBLEMS 


G. W. Gershefski 
Standard Oil Company, (New Jersey) 


WHAT IS OPERATIONS RESEARCH? 

Operations Research is an advanced analytical approach which can aid long term busi- 
ness planning or the solution of daily operating problems. Its main objective is to pro- 
vide management with a scientific analysis of the "cause-and-effect" relationships of a 
specific situation. It is then possible for them to blend this scientific information 
with such intangibles as general business conditions, politics and psychology to arrive 
at a final decision. 


Operations Research can be considered as a method of expanding the number of fac- 
tors that can be considered simultaneously when arriving at a decision. For years, as- 
sistance of this type has certainly been the objective of consultants and of various 
engineering groups. However, the major difference between most of these and Operations 
Research is in terms of the approach and the techniques used. The characteristics of 
Operations Research are that it: 

Applies the "scientific method" to the solution of business problems more 
rigorously. 

Uses advanced mathematical, statistical and computer techniques to aid 
analysis when necessary. 


Let us consider these points in greater detail. 


WHAT IS THE OPERATIONS RESEARCH APPROACH? 

The exact steps followed in investigating a particular problem certainly depend not 
only on the factors to be considered, but also on the type of answer desired, and the 
time available. However, the approach outlined below, as an example, is typical of that 
which has already been used to solve several Operations Research problems. These five 
steps will be described in turn: 

1. Define problem and set limits and objectives. 

2. Collect data and analyze. 

3. Develop mathematical model of the situation. 

4. Test the accuracy of the model. 

5. Use model. 
Although the 0.R. approach is very general and can be used for many problems, the dis- 
cussion revolves about a terminal planning problem to aid continuity and understanding. 


1. Define Problem and Set Limits and Objectives 

While it may appear to be obvious, the thoroughness and clear thinking used in this 
first step often can determine the eventual success of the project. Once having looked 
into the matter of what is required and the ability oi Operations Research techniques to 
handle the job, boundaries must be defined so as to determine the scope of the project. 
It is here that preliminary investigations and conferences with management take place. 
Once the problem is clearly defined and the scope set, the other steps can follow in se- 
quence. 


A typical terminal planning problem might be stated as follows: 
Develop a planning tool that would depict comprehensively the numerous 
variables involved in the terminal operation so as to assist in deter- 
mining (1) the "best" number of berths and (2) most efficient amount of 
terminal tankage. 


2. Collect Data and Analyze 

In a problem of this type, extensive interviews of terminal personnel must be made 
to determine operating characteristics, tank and piping arrangements, and special prob- 
lems associated with the terminal operation. Data would also be analyzed to determine 
the characteristics of tanker arrivals, berthing times, and loading rates. 


3. Develop Mathematical "Model" 

This step (often the most difficult) is concerned with developing some sort of "mo- 
del" which will represent and tie together the various factors involved in the terminal 
operation, so that their cause and effect relationship can be determined. ("Models", 
referred to in Operations Research phraseology, generally take the form of mathematical 
equations and/or computer programs - they are diferent in nature but analagous to 


ASQC LCS Code 
167 


( 

4 
7 

; 


168 1961 ASQC CONVENTION TRANSACTIONS 


to "pilot plants" of refinery or manufacturing processes.) The next section describes 
in more detail the various types of models and how they can be used and constructed. 


The important characteristic of a model is that changes introduced to the model 
must produce much the same effect as when these changes are introduced to the real oper- 
ation. For example, if the model indicates that an increased loading rate of 5000 B/H 
reduces tanker waiting time by 3 hours per day, then in actual practice substantially 
the same reduction must occur. 


The main advantages of having a mathematical or computer model are: 
1. The effect of various changes can be studied without disrupting operations 
or incurring unnecessary construction. For example, the effect of reducing 
loading rates could be studied without incurring the delays that might result. 
2. The answer can be obtained in a shorter time than if actual operating data 
had to be collected and analyzed. 
3. The model can easily be used to see how the business would react under a 
very wide variety of conditions. 


4. Test the Accuracy of the Model 

This step is one of the most important. It consists of determining how well the 
"model" expresses the cause and effect relationship of the variables actually involved 
in the operation. In this case of a marine terminal, one could feed into the computer, 
say - the throughput, arrival and loading conditions of an historical period and compare 
tanker waiting hours obtained through the model against those experienced in actuality. 
If close agreement is not obtained, the model itself must be redesigned. It is this at- 
tempt to approach realism and to avoid unrealistic assumptions that often leads to mod- 
els of some mathematical complexity. 


New models are being developed continually to aid in solving many problems other 
than the terminal problem described here. Typical problems in the marine field being 
considered by Operations Research are: 


AREA TYPE OF QUESTIONS ANSWERED 

Inventory How often should inventories be replenished? 

Control What is the most economic amount to order? 

Terminal What is the best amount of terminal tankage, consider- 
Tankage ing customer satisfaction, capital tied up, and tanker 


waiting charges? 


Tanker What is the lowest cost method of scheduling a fleet, 

Scheduling considering requirements, tanker positions, and tanker 
operating costs? ‘ 

Tanker What is the best way to allocate various sized tankers 

Allocation to different trades, considering operating costs, the 
length of haul, demand, and turnaround? 

Tanker Considering future requirements and chartering costs, 

Coverage what is the best "mix" of owned and chartered ships of 
various types? 

Tanker What is the most efficient tanker size considering oper- 

Size ating costs, tankage requirements, terminal modifica- 


tion costs, and fleet scheduling? 


5. Use Model 

The developed model can be used with confidence to study the effect of changes in 
operating practices or the physical facilities of the terminal. Graphical or other nu- 
merical representations can be constructed showing the effect of throughpvt, number of 
berths, and amount of tankage on tanker waiting time. The job is not complete until re- 
sults have been translated into meaningful and practical form. (See Figure 1). 


This report, in the form of a graph, shows management that with only one berth a- 
vailable at the terminal, waiting time would increase very rapidly when the terminal 
throughput reached a certain point. The effect of a second berth is also shown. (For 
a specific terminal, the actual magnitude of the variables would, of course, be indi- 
cated.) 
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FIGURE 1 
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The important points to note in this process are (and these are not limited to the 
terminal problem described above, but are true for most Operations Research studies): 


A model is developed - using any mathematical or computer technique required - which 
depicts the cause and effect relationship of the variables considered important to the 
problem. The model is then tested. This is where the "scientific" method comes into 
play. 


To be sure, in actual practice, the approach may be modified to adjust for the size, 
scope and ic conseq of any particular problem. In some cases, for example, 
the collection of data and its analysis are sufficient; in others, a more thorough econ- 
omic analysis may be required. 


What Are Some Examples? 

Because of the basic importance of models in really understanding Operations Re- 
search and how it is performed, alternative models which might be used to solve a partic- 
ular problem are described in the pages that follow. 


PROBLEM 
Consider, as an example, a very simple problem: 
Problem: At a marine terminal having only one tanker berth, tankers arrive 


to load crude. What effect will the addition of a second berth 
have on ship waiting time? 


This problem as stated, has been simplified for purposes of illustration. However, 
the basic problem is of real importance to terminal planners. They must know how many 
berths to build so as to handle the anticipated traffic. 


On the one hand, building too many would be extremely expensive and, on the other 
hand, building too few (thus incurring excessive tanker waiting time) would also be ex- 
tremely expensive. 


Conventional methods of solving this problem often center about a "berth occupancy” 
or “use factor." The results of using this basis have not always proved to be satis- 
factory. Operations Research has been able, through the methods outlined below, to ar- 
rive at more accurate answers than hitherto possible. It must be stressed, again, that 
this example is greatly simplified, so as to describe more easily some of the various 
types of models used in Operations Research. 


SOLUTION 1. BY FORMULA MODEL 
The formula below was developed with the use of advanced mathematics. It will hold 
true if the following conditions are met: 
1. A large number of different ships use the terminal. 
2. The tankers arrive without any specific pattern. 
3. All vessels that arrive are loaded on a "first-come, first-served" basis. 
4. Variations in time required to load follow a particular pattern. 


If these conditions are met, the average time that a tanker will be required to 
wait can be determined by solving the formula below. (One of the contributions of Oper- 
ations Research is the developing of formulae to fit business situations.) 


A 
Bowe 
t 
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W = Average Waiting Time per Ship in days. 

= Average Tanker Arrival Rate, Ships per Day. 

=z Average Time to Load, Ships per Day for One Berth. 
c ss Number of Berths. 


The effect of adding a second berth on ship waiting time can be determined by sub- 
stituting the proper valves into this formula model, (first with c = 1, then with c = 2), 
Although this is relatively easy to do, the question still remains as to whether this 
formula is a good "model" of the terminal. The answer will be yes, only if the assump- 
tions concerning the “pattern” of arrivals and tanker loading used in developing the 
formula are met at that particular terminal. 


Actual experience has shown that this formula represents adequately some terminals. 
In other cases a formula may not fit. For instance, if additional variables are intro~ 
duced, (such as restricted anchorage space, limitations on night navigation, docking on- 
ly at certain stages of the tide, etc.), it would be very difficult - perhaps even im 
possible - to develop a formula model which would fit such an involved situation. Alter- 
native methods of solution are Replication models and Monte Carlo models. 


SOLUTION 2. REPLICATION MODELS 

The Replication model works this way: It reviews historical data to determine the 
sequence and timing of events so as to use the past data as an input to e system and then 
note the numerical value of an item required for analysis. To clarify this, suppose 
that the observed data on past ship arrivals and loading time at the terminal were as 
shown in Figure 2. It can be seen in this simple example that the ESSO SANTOS arrived 
at noon on August 2, waited until midnight to start loading at Berth 1, and finally de- 
parted before noon on August 4. The average time a ship spends waiting at the terminal 
can be determined easily by adding and averaging thie past experience. Theeffect of a 
second berth could be determined by using the same arrival and loading data and allocat- 
ing the ships to either one of the two berths as they become available. This new allo- 
cation, incorporating a second berth, is shown in Figure 3. In this case, it can be 
seen that when the £SSO SANTOS arrived it could have gone immediately into Berth 2. A- 
gain, one can estimate the time spent waiting at the terminal and hence evaluate the ef- 
fect on tankerwaiting time of building a second berth. The nice thing about this type 
of model is that it is not necessary to make any assumptions about the pattern of arriv- 
als and loading. Actual past occurances are used to provide this information. If it is 
likely that the same pattern will prevail in the future, and if all pertinent variables 
can be taken into consideration, then this approach can be extremely useful. 


When neither the Formula Model nor the Replication Model apply, the Monte Carlo mo- 
del may provide the solution. 


FIGURE 2 


OBSERVED TANKER ARRIVAL AND LOADING DATA 


(1 BERTH) 
TANKER Arrival and Loading Dato 
3 4 
Cube | 
—Esso-Ponce 


' 
1 
' ' 
otc Time waiting 
{ 
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FIGURE 3 
MODIFIED ARRIVAL AND LOADING DATA 
(2 BERTHS) 
TANKER Modified Arrival and Loading Dota 
Aug | Aug | Aug] Aug | Aug | A 
Esso Cubo ' 
Esso Santos : 
Esso Richmond H 
Esso Montevideo ‘ 
Esso Ponce H 
etc. . 


SOLUTION 3. MONTE CARLO MODEL 

In the above example, the actual arrival and loading data observed were used as a 
basis for arriving at an answer. Under certain circumstances, where, for instance, there 
may be no past experience to draw upon, as would be the case in planning a new terminal, 
or in other cases where extreme complexity is involved, it might be better to develop a 
hypothetical or fictitious pattern of arrivals and loadings. The result of using this 
hypothetical pattern, which may be suggested by experience at similar terminals, will be 
valid if this pattern has the same underlying properties as the anticipated future ar- 
rivals at the terminal. 


For example, one could start by representing the time between tanker arrivals by 
the method used in Figure 4. This figure shows in a convenient way the probability of 
any particular "time-between-arrivals" happening. (This type of diagram is called a 
“Probability Distribution.") In this case, for example, it states that in approximately 
60 cases out of 100, the time-between-arrivals is 12 hours, etc. (Twelve-hour intervals 
were used in this case; in practice any interval could be used.) At this point the per- 
centage of time that an event occurs is known, however the sequence (e.g. 12,12,24 or 
24,24,12, etc.) of those events must be determined. One way of generating a hypotheti- 
cal sequence of arrivals would be to first let the numbers between 1 and 60 represent a 
period of 12 hours between tanker arrivals; the numbers 61 to 90 represent 24 hours be- 
tween arrivals; and numbers 91 and 100 represent 36 hours between arrivals. 


FIGURE 4 


Thus, the numbers from 1 to 100 represent the probability of any particular time 
between arrivals and they conform to the situation shown by the probability distribution 
diagram. Having assigned these numbers to represent a certain time between arrivals, 
the numbers can then be drawn by a random chance - say, out of a hat - to generate an 
arrival pattern. If, for example, the number 47 were drawn, that would indicate a "time 
between-arrivals" of 12 hours. If more numbers were drawn the pattern of arrival in Fi- 
gure 5 might be obtained. In this case, for example, the initial arrival time (an as- 
sumed time used to provide a starting point) was 12:00 on August 1, and the time to the 
next arrival was 12 hours. This places the next arrival at 24:00 on August 1, etc. 


Just as in the Replication model, to solve the problem it is necessary to know both 
the tanker arrival time and time at berth. The time required to load each tanker could 
also be determined by sampling at random from a probability distribution of loading times. 


« ‘ © q 5 
| 
gf 
| 
— ‘ 
50% — PROBABILITY DISTRIBUTION OF 
TIME BETWEEN ARRIVALS 
10% 355555505 
fi 
HOURS 2 24 % i 
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This being done, one has really generated a body of tanker arrival and loading data which 
can then be analyzed in much the same manner as that used for the actual historical data 
by the Replication model (Figure 3). 


The use of chance is justifiable providing a large number of samples are drawn at 
random (in practice, enough to simulate at least a year's operation). The important 
thing to bear in mind is that if the model is correct the combined effec: of all arriv- 
als as generated by chance will be very nearly the same as the combined effect of all 
arrivals occuring in actuality. When viewed in this manner, it becomes apparent that 
the particular arrival time assigned to any one particular ship has no bearing on the 
final answer. Only the combined effect is important. Electronic computers can generate 
a year's pattern in a matter of minutes. Consequently, extensions of this type of ap- 
proach have proved to be very valuable in the actual analysis of terminal problems at 
Esso Pet's Fawley Terminal and Creole's Lacustre Terminal. 


It should be noted here that a model of an operation need not involve a computer 
program - in this example, a formula, capable of being solved manually, might suffice. 
The need to translate a model into a computer program depends on its complexity and the 
use to be made of it. The computer is only enother tool to be used when advantageous. 


FIGURE 5 
GENERATED TANKER ARRIVAL PATTERN 
(ASSUMED STARTING TIME: 1200, AUGUST 1) 
Number Time Time of 
Drawn ot Between last Arrival 
Random Arrivals Arrival Time 
36 12 | 1200,Aug1 | 2400, Aug 1 
80 24 2400, Aug | 2400, Aug 2 
25 12 2400, Aug 2 1200, Aug 3 
31 12 | 1200,Aug3 | 2400, Aug 3 
etc. . 
TYPICAL FACTORS SPECIFIED AS INPUT TYPICAL OUTPUT OF MODEL 
Arrival Pattern of Tankers Tanker waiting hours by class of 
Tanker Sizes vessels and caused delay 
Tanker Loading Rates Required changes in tanker alloca- 
Berths tion slates 
Berthing Time Crude shut in 
Size and Crude Allocation of Crude degradation required 
Each Tank Inventory level, all grades 
Fill and Suction Line Manifolding Number of times a tank is swung 
Crude Metering Facilities over from one crude to 
Crude Production Rates another 
Pipeline Pumping Capacity Berth utilization 
Field Tankage Line utilization 


Models can be used to solve many complex problems where numerous variables are in- 
volved. By way of example, the above are typical of the factors which were actually con- 
sidered in one Monte Carlo model. (The reader will recognize that the problem is much 
more advanced than the one described above.) 


WHICH MODEL TO USE? 

The problems discussed above have been fairly simple, the approach straightforward. 
In reality the way of approaching a problem and determining the type of model to use de- 
pends on the problem or business to be simulated (size of operation, number of variables 
tnvolved, amount of variation present, day-to-day operating or planning use?), the a- 
mount and reliability of data (sufficient to study past behavior, changes and inter-re- 
lationships?), the type of answer desired (a "best" solution, a case study tool, or use- 
ful graphs?), the availability of computing facilities (can complex formulae be easily 
evaluated?), and the suitability of various mathematical formulations (allocation prob- 
lems solved by Linear Programming ; inventory problem-solved by Markov Theory; Conges- 
tion-solved by Queuing Theory; Scheduling problem-solved by Network Analysis; Competi- 
tive problem-solved by Game Theory; etc.) 


the 
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The essential point is that changes introduced to the model must produce nearly the 
same effect as if they were introduced to the business situation itself. 


SUMMARY 
In summary, Operations Research is an analytical approach which can often aid in 
the solution of business planning and operating problems. The chief charteristics are: 


A scientific analysis is made of the cause and effect relationship of 
many of the important variables in a problem. This analysis tends to 
narrow the range of “intangibles” that have to be considered by manage- 
ment in making a decision. 


The latest advances in mathematics, statistics, and computer technology 
are used to develop a model of the business. Recent advances in these 
fields make the approach especially useful when there are many variables 
to consider and/or these variables, themselves, vary over a wide range. 
Specifically, modern electronic computers can solve in a matter of minutes 
mathematical problems which, when attempted by hand, might take days. 


The approach is often concerned not only with getting "an answer" to a 
problem (many answers may be feasible), but also a basic understanding 
of the operation and the variables involved so that the "best" answer 
can be obtained. 


Although this is a relatively new field, many successful applications have already 
been made. As time goes on, people working in Operations Research are developing both a 
firmer understanding of the nature of business problems, and of the capability of vari- 
ous types of models to cope with the problems presented. The result of this activity 
has been an increased ability to aid management by providing factual, objective analyses 
as a basis for policy-making decisions. 
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THE HUMAN EQUATION IN QUALITY CONTROL 


Glenn Gilman, Employee Relations Consultant, Lockheed Aircraft Corp., 
and Professor of Industrial Management, Ga. Inst. of Technology 


Always, we come back to the need to control people if we're going to control 
quality. And as science and technology become more and more the handmaidens of 
quality production, so does thehuman equation appear very often to be more and more 
recalcitrant! Perhaps this isn't so -- perhaps science and technology have not 
1 d our d ds on people as mich as we're inclined sometimes to think they have -- 
but we'll come back to this later. 


I'd like to stay for a moment with this idea of "controlling people. Taken 
literally, the term represents a dangerous false assumption, We don't "control" people 
in any direct way. When we attempt it, we're likely to run into those frustrating 
experiences that encourage the use of such adjectives as "stupid", “stubborn”, "hammer- 
headed", “insensitive"’ -- but you can finish out the list. Most of us have already 
discovered that if we wish to change someone's behavior, our best bet is to change 
certain aspects of the situation in which he lives and works, to his interpretation of 
which he responds. If he moves in the direction we wish him to go, as a result of our 
"control", it's because we have managed to make a significant alteration in his life 
situation as he sees it. 


Controlling human behavior is not merely a matter cf introducing a significant 
change, of either a positive or negative nature, into an individual's life situation, 
however. Let me recall the phrase we just used: "as he sees it". Each of us looks at 
the world through a scanning glass that's polarized and to some degree distorted by 
feeling and emotion, value judgments, and the indelible traces of past experience. The 
individual whose behavior we wish to change does not respond directly to the situation 
we have recast any more than he responds directly to us, He responds to his interpre- 
tation of what he sees. He may not see what we wish him to see, or if he does, he may 
quite possibly attach a meaning to it that we never intended. 


It may be necessary for us to help him increase his sensitivity in one area and 
reduce it ia another, help him re-evaluatethe meaning of his past experiences, and help 
him modify his value judgments before the desired change in his behavior comes about. 
Having done all this, we may not find it necessary to introduce change into his environ- 
ment at all <= the desired stimuli may always have been there, but either he has not 
seen them, or he has been interpreting them incorrectly. Let's remind ourselves that 
even in this case his changed behavior remains a function of his perception of his 
situation as he sees it. 


THE SOCIAL CLIMATE FOR QUALITY CONTROL 

I'm sure all of us will agree that the proper way to achieve high quality is to 
make quality production a source of job satisfaction to the man on the production line. 
A number of people are inclined to believe this ought to come about "naturally". In 
terms of our comments of a moment ago, they feel that the average work situation already 
embodies all the necessary positive motivation for high quality production; and that 
when the worker fails to respond, it is due to some inadequacy on his part. "Pride in 
workmanship is dthing of the past" they remark sadly. The typical production worker 
today, they tell us, far from having any real interest in quality, is concerned only 
with doing work that's just good enough to get by. 


The truth of the matter is that there is probably a higher proportion of people 
doing high quality skilled work today than there ever was == for in recent years we've 
added the technician to the “old aristocracy" of the craftsmen, and we haven't reduced 
the proportionate number of the latter in the work force, What has happened is this. 
As a nation, we've so increased our demands for quality production during the past two 
decades that we can now meet them only through mass production; and we have begun to 
ask the semi-skilled worker, the man on the assembly line, to pay the same attention to 
detail and quality that we formerly expected only of skilled workers. 


But we can't expect that he will do it "naturally". In the first place, he has 
not had the benefit of the rigorous training of the craftsman that results in a sense 


of "profession" and built-in standards of quality which demand satisfaction from within 
himself, In the second place, assembly-line work does not involve him sufficiently on 
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a personal basis for production to be naturally a matter of "personal" concern to him. 
In the third place, while we have actually increased our demands for quality workman- 
ship, we have tended to decrease the emphasis on it that reaches the public <= which 
includes our semi-skilled worker, 


This last item may seem paradoxical, but there are explanations for it. In many 
cases days, the ts that call for quality wortmanship in a consumer goods 
item are so complex that we're afraid the buying public won't understand what we're 
talking about if we emphasize the quality side of the picture. We dwell on ease of 
operation, styling, and emotional appeals to sell our product for us. Quality is as 
important as it ever was; perhaps more important, for customers 4emand reliability from 
an increasingly complex product <= but we don'tstress it in the consumer goods market, 
We obscure it through emphasis on style change that is intended to make the customer 
unhappy with what he has and eager to discard it in favor of what is perhaps the same 
basic product dressed up in a new package. We should not be surprised then if our semi- 
skilled worker, an important member of the consuming public, appears to be more con- 
scious of the “wrapping” than of the quality impiications of what's inside. 


All this by way of pointing up the fact that if we wish our production workers to 
“build quality into the product" for us, we've got to do more than tell them what we 
want and rely on an inherent and natural tendency on their part to give it to us. 
Neither their individual backgrounds, mass production methods, nor the prevailing 
social climate encourage a "natural" interest in quality production. So the "human 
relations challenge" in Quality Control is this: 


“How can we make the production worker quality-conscious? 


This is a question of motivation; so let's turn our attention to this topic for 
a few moments, 


THE BASIS OF MOTIVATION FOR QUALITY 
"Motivation" is a word we're likely to use without being sure other people under~ 
stand it to mean the same thing we do; so let's get ourselves in step with a definition. 


Motivation is a socio=-psychological mechanism that "moves" us to satisfy a certain 
need in one way rather than another, in such a fashion that each step toward satis- 
faction is a meaningful step. 


Translating this definition into the quality control picture, the answer to the 
question we raised above becomes quite obvious. 


We make the production worker quality-conscious by making quality meaningful to 
him, a matter of importance, as a part of his progress toward the satisfaction of 
a&_basic personal need 


As the basic need concerned, let's establish "job satisfaction" -~ which we will 
take to mean not only the worker's paycheck, but other less tangible returns he may 
ask of his job. These might include 

1. A feeling that he is "part of the team", arising out of his knowledge that 

he's doing useful work that is adequately recognized. 

2. <A sense of understanding and being understood, 

3. A sense of self-respect. 

4. Work that is "interesting". 


Obviously, when these and other satisfactions are lacking or are difficult for 
him to obtain, his paycheck becomes overwhelming in its importance to him. It has to 
substitute for all the rest. Perhaps it's not quite so obvious -- but when we in man- 
agement assume that his paycheck is all that matters to the worker, our assumption is 
quite often "proven" by his behavior. Not b the mption is basically true -- 
but because simply by virtue of having made it, we frequently fail to provide him with 
the opportunity to realize other satisfactions! 


POSITIVE AND NEGATIVE APPROACHES TOWARD MOTIVATION FOR QUALITY 

We can choose between two basic approaches in making quality a matter of personal 
concern to the worker as he seeks job satisfaction. 

(1) We can impose "quality" as a hurdle, a barrier, between the worker and his 

paycheck, 

In doing this we are, by implication, assuming that his paycheck is all that 
really matters to him, We're making quality important to him by introducing it as a 
threat that must be overcome if he is to reach an ultimate goal of personal importance. 
The "control" -- the motivation for quality -- remains almost entirely external to the 


: 
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worker. Any lowering or withdrawal of the “inspection hurdle" will result in an 
immediate flood of poor quality work. It's an expensive method, since it requires 
almost constant surveilance of worker activity; and it can never be completely effec- 
tive since (as we've already noted) the employer no longer has sufficient control over 
the worker's chance to earn a living to give an economic threat much more than nuisance 
value. 


And since we've implicitlyestablished his paycheck as his only important goal, 
there's a good chance that when we use this method we will fail to provide the worker 
with other opportunities for job satisfaction that might encourage quality production 
in a more positive way. 

(2) We can establish "quality" as a positive rather than a negative factor in 
the worker's mind, encouraging him to look upon it as a stepping stone toward, rather 
than a barrier in the way of, job satisfaction. We may even be able to get him to see 
it as representing one aspect of job satisfaction in itself! 


Under this second method, we hope we may get the worker to “internalize” quality 
standards so thoroughly that they become as much as part of his "self" as his prefer= 
ence for fried chicken over snails, or his political allegiance. We want him to turn 
out quality work as a matter of "self" satisfaction, even though he's aware that it's 
not going to be inspected. 


I'm sure we'll allagree that the second method is the preferable one. Its advan- 
tages are obvious -- "100% inspection", a monitoring rather than a "policing" role for 
Quality Control, a generally better morale climate for production, and so on, But 
agreeing that a method is desirable doesn't get it into operation. We're still faced 
with the question we raised a few moments ago -- how do we go about establishing posi- 
tive motivation toward quality as an integral part of the make-up of a production 
worker? 


SOME STEPS TOWARD POSITIVE MOTIVATION FOR QUALITY PRODUCTION 
We'll merely note for the record that production management has considerable res- 
ponsibility in this area as well, and forget about it. What can we in Quality Control 
do, both directly and in "backing up" Production, toward making quality appear to the 
worker as a stepping stone rather than a barrier -- as a goal in itself, rather than 
an obstacle to be surmounted on the way toward reaching a primarily economic objective? 


Some suggestions follow. They are not all-inclusive by any means. Theirpractical- 
ity may vary from one situation to another. But perhaps they will at least point us in 
the right direction --or encourage you to look for the right direction in the process of 
refuting them! 


1. Let's keep Quality Contro! in its proper functional perspective. 

As staff people, we must guard constantly against the inevitable tendency to think 
of our delegated function as our principal objective. We want to be considereda "part 
of the team" -- then let's think that way, and what's more important, be sure our people 
think that way. The Quality Control executive or engineer is at a level where it's not 
too difficult for him to see the "big picture", and realize that Quality Control is 
only a part of it. We tend to forget how difficult it is for our people in the lower 
echelons to maintain this perspective. 


But if we are successful in reaching and maintaining an “end-product orientation" 
for ourselves and our people down the line, we will have taken the first and most im- 
portant step toward selling the production worker on quality as a personal goal. If 
we can convince him that we really think of ourselves as helping him to build a quality 
product, we will have identifed ourselves with him. We will have set up the rapport 
that must precede anything else we may want to accomplish in a positive direction, 


2. Let's estatblish realistic standards for quality. 

We're going to convince the worker of the importance of quality workmanship only 
when the standards for judging it are realistic from two points of view: 

(a) The functional need for them, 

(b) The physical possibility of meeting them. 


We're going to have to justify quality standards to the worker in terms of the 
"sense" they make to him, This is tough enough even when they're necessary, Let's 
never find ourselves in the unenviable position of trying to justify standards that 
are actually arbitrary and unreasonable, merely to "save face". We're net likely to 
succeed in the attempt; and we'll have created an atmosphere of suspicioa and distrust 
that will cast doubt on the remainder of our standards that are functionally necessary. 
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Faced with standards that are physically impossible to meet, the worker is not 
likely even to come as close to them as he can. More usually, he will stop at a level 
of quality that is less than he can turn out, simply to emphasize to us how impossible 
the standards are. On the other hand, when it is merely difficult to achieve them, but 
we have convinced him of the functional necessity for doing it, the worker can regard 
high standarde as a challenge, and our insistence that he meet them as an important and 
necessary part of production, 


I'm aware that economic considerations alone are likely to keep us from deliberate- 
ly setting standards higher than they need to be. But I'm also aware that we're not 
always as sure as we should be how high they do need to be, and the tendency is to "play 
it safe", particularly if we happen to be an hourly rated inspector. When the produc- 
tion worker asks "how much is enough?", we ought to be in a position to answer him on 
the basis of sound reliability research. 


3. Let's educate the worker for quality. 


It isn't enough to establish realistic quality standards from a functional point 
of view. We've got to explain to the production worker WHY they are functionally neces- 
sary. Though the quality of the end product may not be important to him as the eventual 
user, he possesses the human ability to identify with the user; and through this empa- 
thetic relationship quality can become a matter of concern to him even from the stand- 
point of use. This is one of the most important of the ways by which he “internalizes" 
quality standards. 


If, for reasons which we must admit are not completely under theinfluence of 
Quality Control, the worker identifies in a positive fashion with the company itself, 
we have a second road to "internalization". As we are able to justify the need for 
quality from the company's point of view, he makes it a need from his point of view. 
While this possibility is a function of company-employee relations in general, it 
becomes apparent that whatever Quality Control can do to elevate the general level of 
morale in the plant will have a direct bearing on the quality of output. In this res- 
pect we ought also note that deliberately poor quality is one of the weapons an employ- 
ee may use in "fighting back” at a company when he feels himself to be poorly treated. 


This education of the worker as to the functional need for quality ought to be a 
part of his initial orientation, of his initial and subsequent formal training, and a 
continuing on-the-job process, It is best handled by the Training Department and the 
"line" -- with the backing and cooperation of Quality Control, of course -=- not neces- 
sarily because these people can do it better, but because we want to impress on the 
worker the fact that quality is a matter of impertance to everyone, not merely to the 
people who have been delegated the specific function of "controlling" it. Quality 
Control people must be prepared to do an "ad hoc" educational presentation at any time, 
however, as it becomes necessary or desirable to explain to people in Production the 
“why” that is behind a particular set of quality requirements. 


4. Let's train for quality 
If Quality Control is going to maintain the respect of the production worker, the 


inspector has to know considerably more about the component and its use than is needed 
merely to compare it against a set of arbitrary standards, Even when inspection is a 
"go-no-go" proposition, he must be able to justify the standards in terms of the func- 
tion of the component. When inspection is not this "simple", and borderline cases are 
encountered that call for the exercise of judgment, his decision will be accepted as 
proper only when he can go beyond "what the book says", and explain it in terms of the 
component in its total context of function and use. 


He will be called on not only to exercise judgment himself, but to recognize its 
capable exercise by others. Most difficult of all, he will need to recognize his own 
limitations, and know when to call in a superior or a staff man who is better qualified 
than himself to pass on a case in the "gray" area, 


Though Quality Control is not responsible for the training of production person- 
nel, it can make its job later on considerably easier by cooperating in every way 
possible in the initial training process. Here again, a “behind-the-scenes" role is 
best from a strategic standpoint, but the occasional appearance in the training pic- 
ture of an inspector or Quality Control representative who can demonstrate that he and 
his brethren are in the plant to help the worker rather than hinder him can pave the 
way for positive relations later on at the workplace. 


It is through training of the production worker, nS that a third ave- 
nue is opened by which he can “internalize” standards of quality. He will never 
incorporate then as a part of the disciplinary mechanism of his "self" if he cannot 
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achieve them. Quite the contrary <-- in order to protect his image of himself as com- 
petent and capable, and to avoid painful self-criticism, he will convince himself that 
they are arbitrary, unrealistic and unimportant. But once he has acquired sufficient 
skill to work to standards comfortably, he is in possession of a very effective "“ego- 
building" device. If other aspects of his work situation are handled properly, the 
achievement of quality production can become a job satisfaction in itself. 

5. Let's maintain rigorous enforcement of reasonable and uniform standards. 

In the final analysis, the worker believes what he sees rather than what he hears. 
Unless Quality Control itself demonstrates that it believes quality to be of sufficient 
importance to call for rigorous and uniform enforcement of the standards it has estab- 
lished, we can'texpect the worker to get very excited about it. Let's make this clear— 
we have not been recommending a "soft" attitude on the part of inspection. Reasonable, 
yes; but because of the very reasonableness of the standards we have established, be- 
cause of the very reasonableness of the standards we have established, because of the 
education we have sponsored with regard to the need for them, and because of the train- 
ing we have provided for their achievement, we cannot only dare to be but mst be un- 
yielding in our demands that they be followed in every particular as we have set them 
up. 


Lack of uniformity in application of quality standards not only weakens our case 
as we plead their necessity; it destroys the effectiveness of the whole approach we've 
been suggesting. We want to build quality into the product rather than inspect it in, 
This can be done only when the importance of quality is consistently and uniformly em 
phasized all along the production line, rather than merely at certain designated inspec- 
tion points, 


6. Let's get our people "snapped in" on human relations. 
We started out with the premise that Quality Control is essentially concerned with 


influencing the behavior of people. We've emphasized that this is effected not by 
exercising “control” over them directly, but by bringing about certain changes in the 
situations to which they respond, We want to do this in such a fashion that it results 
in a realignment of their personal value systems -- so that, in effect, they themselves 
generate a "climate" that is favorable to the production of a quality product. 


From this point of view, the value of sound training in human relations for 
Quality Control personnel becomes apparent, They needn't be social scientists -- but 
they need to know more about human behavior than the layman ordinarily does. The good 
inspector must be able to criticize a man's work without putting him on the defensive, 
for example, A few of us can do this without benefit of training; but most of us can 
do a mch better job of it once we understand how man forms his image of himself, how 
important that image is to him, and the lengths to which he will go to protect it when 
he feels it is threatened. 

7. t's make the Quality Control function "authoritative" rather than "authoritarian". 

We all recognize a potentially explosive situation in the relationship between 
the hourly rated inspector and the salaried supervisor, simply because of the power 
which the inspector must have if he's going to be able to do his job. It seems to me 
there are two parts to the only practical answer to this problem, The first part we've 
already outlined-- if we select our inspectors carefully and train them properly, the 
authority they possess on a functional basis will be defined as "that to which we look 
for an answer", rather than as "that which gives one the power to command", The "com- 
mand" which they exercise will be fundamentally that of respect -- the same kind of 
respect that is accorded by the salaried man, whether he is a supervisor or not, to any 
craftsman who knows his job, This, it seems to me, is the most important kind of 
"backing" we can give the inspector, whether he be hourly rated or salaried <= the 
preparation he needs to do his job properly. 


The second part of the answer -- and the second kind of backing -- is to be sure 
that our Quality Control supervisors and their inspectors are “teams” in a truly func- 
tional sense, I'‘mreferring now not only to the usual kind of relationships that ought 
to exist between any supervisor and the people in his work group, but as well to the 
degree of functional integration and reinforcement the Quality Control supervisor is 
able to give his inspectors. He must operate not only as an administrator, but as a 
Quality Control engineer -- does he have the time as well as the training to do what 
needs to be done in this respect? If the answer is "no", will additional training in 
administration help him to improve his organization of this phase of his work suffi- 
ciently to free him for the needed time out on the floor? 


If the answer is still "no", ought we give him some additional help? -- perhaps a 
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clerk to take the load of paper work off his shoulders, or perhaps a salaried techni- 
cian whose only responsibility is the functional performance of the group? Perhaps 
even more extensive revision of the administrative structure of Quality Control might 
be helpful -- you know the answer to this better than I. What seems important to me is 
that the Quality Control supervisor or his representative be not only qualified to pro- 
vice his people with authoritative functional back-up, but that he be in a position to 
be able to do so. 


This is important not only with regard to the quality and uniformity of the job 
Inspection does -- it has, as well, a salutary effect on the relationship between the 
inspector and the supervisor. When the hourly inspector knows that support is imme- 
diately available for him if he needs it -- he is less likely to need it! His whole 
approach to the supervisor becomes altered. He is less likely to let emotion color 
his presentation if he knows that rescue is at hand when he seems to be getting beyond 
his depth; and as a result his effectiveness is increased rather than diminished. 


SUMMARY 
Actually, we haven't said anything more than this -- the most important thing to 


keep in mind about the “human relations chalienge of Quality Control" is that we are 
exercising the control with regard to people rather than things. 


When we keep this in mind, just plain common sense will do the job for us. There's 
no mystery involved, once we understand the real nature of our objective. 
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BAYESIAN DECISION THEORY AND STATISTICAL QUALITY CONTROL (2) 


Morris Hamburg, Associate Professor of Statistics ‘?) 
University of Pennsylvania 


Bayesian decision theory has had considerable development in recent years and its 
impact on the field of quality control is being increasingly felt. This type of 
theory emphasizes the importance of including economic costs and prior knowledge for- 
mally in the solution of a problem involving the making of decisions under uncertainty. 
Since quality control problems virtually always involve decision making with the ele- 
ments of costs, prior knowledge and uncertainty as essential ingredients, the develop- 
ment is a very logical one. It is an interesting point, however, that although there 
have been quite a few articles published on the general subject of Bayesian decision 
theory applied to quality control problems, relatively little of this material has 
been published in periodicals sponsored by the American Society for Quality Control. (3) 
As a matter of fact, in an unpublished bibliography by I. R. Savage on articles “in 
which the economic aspects of the quality control program are emphasized," only five 
of the ninety-eight items listed appeared in ASQC sponsored publications. Only one of 
the five articles actually imvolved Bayesian decision considerations. Perhaps a par- 
tial explanation is that a substantial part of the development of Bayesian decision 
theory applications in quality control has taken place outside this country. 


The purposes of this paper are to indicate the basic elements of the Bayesian de- 
cision theory approach in a quality control setting and to summarize briefly some of 
the criticisms this theory makes.of traditional estimation and hypothesis testing pro- 
cedures. A few brief remarks on the historical development of some of the pertinent 
ideas will be given before proceeding to the critique of classical statistics. 


Reverend Thomas Bayes, an English minister who lived in the eighteenth century, 
developed a theorem which permitted the calculation of the probabilities of "causes" 
when evidence was available concerning "effects". Specifically, Bayes showed how to 
calculate posterior probabilities °f occurrences of events given prior probabilities 
of the occurrence of these events. ) Paradoxes arose in connection with the early 


(1) asge Les Code 400:00:000 


(2)7 am greatly indebted to Professor Howard Raiffa, Director of the Ford Foundation 
Institute of Basic Mathematics for Application to Business and Professor at the 
Harvard University Graduate School of Business, who inspired my interest in 
Bayesian decision theory when I was a fellow at that institute in 1959-1960, 


(3), notable recent exception to this point is the set of papers by D. R. Cox, A. Hald 
and G, B. Wetherill which appeared in Technometrics, Vol. 2, No. 3, August 1960, 


(Wn Bayes' original work, he assumed equal a priori probabilities. The theorem 
known today as Bayes’ rule which does not require the assumption of equally probable 
prior events is actually La Place's modification of Bayes' original theorem. The 
theorem gives the posterior probability of the event Ay, which is one of a partition 
of events Ay, i=l, 2, ..., m. The appearance of one of the Aj is a necessary condi- 
tion for the occurrence of another event B, Knowing the conditional probabilities 
P(B/Ay) and the unconditional probabilities P(Aj), the posterior probability of Ay 
given that B has occurred is 

P(Ay)P(B/Ay) 


P 
P(Ay)P(B/Ay) 


is 


the question, Given that event B has occurred, what is the probability that it was 
preceded by the event Ay? Another way of looking at it is that P(A,y/B) is the re- 
vised probability assignment to event Ay after observing evidence B. Bayes' paper 
"An Essay Towards Solving a Problem in the Doctrine of Chances" was submitted post- 
humously to the Royal Society in 1763, A reprint of the essay appeared in 

Biometrika, Vol. 46 (1958). 


The theorem answers 
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applications of the theorem because of the use of the "Principle of Insufficient 
Reason." That iy, it was assumed that when the a priori existence probabilities were 
unknown, they could be taken to be equal. When the prior probabilities are know, 
whether equal or unequal, there is no difficulty in the application of the theorem. It 
is probably safe to say, however, that the major obstacle to its widespread application 
has been the difficulty of determining the a priori probabilities. Modern Bayesian de- 
cision theorists, however, hold that legitimate applications of the theorem can be made 
not only when prior probabilities are known in the objective relative frequency sense 
but also when they are determined on a "subjective probability" or "personalistic prob- 
ability" basis reflecting the a priori caps = of the investigator concerning 

the underlying "events" or "states of nature". 5 


There were a number of discussions of Bayes' in connection with statisti- 
cal quality control early in the history of that rica (by Me — Shewhart, Simon and 
various articles in the Bell System Technical Journal, ( To eins just one example, 
Leslie E, Simon developed a very interesting set of nomographs based on the 
Laplacian generalization of Bayes' formla, 


w(x) x°(1-x)9-Cdx 
el(n-c)! 


W(Qr,Qy) = 


x) x°(lex)®~Sax 
el(n-c)} 


where W(Qr,Qu)is the a posteriori probability that the proportion of defectives in a 
lot lies between the limits Qrand Qu, W(x) os the prior probability distribu- 


tion of lots with proportion defective Qex and = ae ; x°(1-x)"-© 4s the binomial 


probability that the lot would produce the observed sample of n articles c of which 
are defective. Simon assumes W(x) is constant, that is, equal likelihood of lot frac- 
tion defectives prior to sampling. However, he gives a very interesting discussion of 
the effects of the prior distribution of lot fraction defective, sample size and sam- 
ple fraction defective on a posteriori probabilities. The nomographs provide both a 
posteriori parameter estimates and a priori probability calculations. It seems sur- 
prising that wider application has not been made of Simon's early work, not only of 
the Bayesian concepts, but also of the techniques and ideas developed in his "grand 
lot schemes", One would only need to add economic costs to Simon's Bayesian mathe- 
matical model to have a quite modern formulation. 


Turning now to a brief critique of classical statistics, let us consider first 
of all the case of confidence interval gy*inaticn. A typical illustration in a qual- 
ity control text might run as follows, Take a simple random sample of 10 articles 
from a large lot. If no defective articles are observed in the sample, the 0.95 con- 
fidence limits for the lot fraction defective are 0 to 0.31; if one defective article 


OO) see L. Je Savage, The Foundations of Statistics, John Wiley and Sons, 195i for a 


discussion of the jectiv c view o ility. 
(6)see L. E, Simon, An Engineer's Manual of Statistical Methods, John Wiley and 
Sons, p. 166. 


(8) se6 ®.€. Acheson J. Duncan, quality Control and Industrial ‘Statistics, Richard 


D. Irwin, Inc, Homewood, Illinois, » Pp. 427-40, 
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is observed, the 0.95 confidence interval is 0 to 9.45, etc. It is usually pointed 
out (correctly) that these are not probability statements concerning the population 
parameter because the »opulation parameter in the classical approach is not a random 
variable. As a matter of fact, in classical statistics, probability statements gen- 
erally concern conditional probabilities of sample outcomes given specified population 
parameters. The Bayesian point of view would be that these are not the conditional 
probabilities we are usually interested in. Rather, we would like to have the very 
thing not permitted by classical methods - conditional probability statements concern- 
ing population values given sample information. With the use of appropriate prior 
distributions, posterior probability statements can, of course, be made about universe 
parameters considered as random variables. The illogicality of neglecting prior infor- 
mation in making estimates of universe parameters was brought out well, I believe, in 
a comment of a late colleague of mine to the effect that it had always bothered him 
that he could make as good a confidence interval estimate in an area he knew nothing 
about as could a statistician of equal competence who had spent his life working in 
that area. This suggests the frequently made point that often the classical statisti- 
cian faced with a decision to make will act in a way that indicates that he does not 
take completely seriously the inferences dictated by classical methods. 


An illustration of a situation in which one would not be paying te act om a con- 
fidence interval estimate neglecting prior information follows. Suppose that a 
large lot of articles had been produced and by means of 100 percent inspection it was 
determined that 80 percent of the articles were "good" according to specified stand- 
ards. Some of the defectives were subsequently reworked and rectified, but through 
poor management and bad record keeping the identity of the good and defective articles 
was lost. The lot was then placed intact in a suitable storage area for a very brief 
period during which no deterioration took place, Prior to shipment to the customer, 
management decided to take a sample of articles from the lot and to set up a confidence 
interval estimate for the percentage of good articles. A sample was drawn and al- 
though the rules of random sampling were followed, there were a sufficient number of 
defective articles concentrated in this particular sample to i ld a 0.95 confidence 
interval estimate of 65 percent to 75 percent good articles. ( 0) Since it is know 
that at least 80 percent of the articles in the lot were good, (as a matter of fact 
there is a reasonable presumption that the percentage is substantially higher than 

80 percent), the confidence interval estimate cannot be taxen seriously for a conclu- 
sion about the quality of this lot. However, the point of this illustration is that 
classical statistics permits the inference to be drawn only from the confidence inter- 
val statement and idence coefficient (or decision rule) mst be selected before 
examining the sample evidence. In the Bayesian approach, a prior distribution of per- 
centage of good articles which had a lower limit of 80 percent would have forestalled 
an inference such as the one given above since the posterior distribution would then 
have had a corresponding lower limit of 80 percent. 


Turning now to a second illustration, the essential ingredients of the Bayesian 
approach will be indicated by means of a simple hypothesis testing example in an ac- 
ceptance sampling setting. (1 ) This example will also illustrate some of the elements 
of the critique of traditional hypothesis testing. In particular, it will demonstrate 
that if tests of hypotheses are conducted in the usual manner of setting decision 
rules of rejecting or failing to reject hypotheses at pre-set levels of significance, 
non-optimal decisions may be made. 


(nis example is similar to one given by Schlaifer, R., in Probability and Statistics 
for Business Decisions, McGraw-Hill Book Co., Inc., N. Y. 1959, pp, LEEBEE. 


(10) following quotation at the head of Chapter 6 in Leslie E, Simon's, An pugineer’s 
Manual of Statistical Methods covers the situation of the sample in which fa s 
been unkind to the sampler: 


"Alle Kunst is umsumst 
Wenn ein Engel auf das Zindloch brunzt." 
= An Old Folk Saying 
Done idea for this example was derived from a similar illustration involving drill- 


ing alternatives in the oil industry in an unpublished note written by Professor 
Howard Raiffa. 
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Assume a situation where a manufacturing company does inspection on a lot by lot 
basis for a particular product prior to the movement of these lots between two depart- 
ments of the company. Acceptance sampling inspection is carried out by selecting a 
single random sample of n articles from each lot. Further assume that there are only 
three different types of lots produced, namely 2 percent, 5 percent and 7 percent de- 
fective. The manufacturer wants to accept and send to the next stage of oroduction 
lots that are 2 percent defective but he wants to reject and rework lots that are 5 
percent or 7 percent defective. In decision theory terms, therefore, there are three 
"states of nature" or possible "events" - 2 percent, 5 percent and 7 percent defective 
lots, and two terminal acts - "accept" and "reject". A payoff matrix is given in 
Table I, showing the costs incurred under the various combinations of events and ac- 
tions. If a "bad" lot is accepted, the losses due to repairs which become necessary 
at later stages of production are respectively $300 if the lot is 5 percent defective 
and $500 if the lot is 7 percent defective. If a "good" lot, that is, a 2 percent de- 
fective lot, is rejected, the losses attributable to seeking out defective product 
and the making of unnecessary repairs has been established to be $200. All other 
costs are shown as $0, It will be noted that the relevant costs in this type of prob- 
lem are “opportunity losses", that is, the additional losses incurred by not making the 
correct decisions with respect to lots submitted for inspection, 


TABLE I 
PAYOFF MATRIX SHOWING OPPORTUNITY LOSSES FOR ACTIONS OF 


ACCEPTING AND REJECTING LOTS WITH 
SPECIFIED PERCENTAGES OF DEFECTIVE ARTICLES 


Event Act 

Lot 

Defective Reject Accept 
2 $200 $0 
5 $0 $300 
7 $0 $500 

Expected Losses $1.00 $200 


On the basis of past production performance it has been determined that 50 per- 
cent of the lots produced are 2 percent defective, 25 percent are 5 percent defective 
and 25 percent are 7 percent defective. These past relative frequencies, in the ab- 
sence of any further information, are adopted as the prior probabilities that 2 per- 
cent, 5 percent and 7 percent defective lots will be produced and submitted to the 
given sampling plan for acceptance or rejection. (Table II). 


TABLE II 


PRIOR PROBABILITIES (RELATIVE FREQUENCY OF OCCURRENC:) 
OF LOTS WITH SPECIFIED PERCENTAGE DEFECTIVE ARTICLES 


Event 


lot Prior 
Defective Probabilities 
2 50 
5 225 
7 225 
1.00 


Formally, the assumption is that the prior distribution of the probability p of a de- 
fective item is the mixed binomial distribution 


3 
(ai, Pi), (i= 1, 2, 3) where =l 


Expected losses for the two actions of acceptance or rejection are shown in Table I, 
These are mathematical expectations of loss under each action if no sampling is done 
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and are obtained by summing the products of the prior probabilities and losses under 
each action, Although expected monetary values are used throughout the illustration 
given here, it may be noted that in situations where they are not appropriate guides 
to decision making, the analysis should be carried out in terms of expected utility 
values. 


A" game tree"is given in weno 1 which serves to indicate the various actions 
that may be taken and their - Our first choice is ery,’ pow? 
ple and inspect m items from the cubmitted lot or not to sample nor inspect. 12 Sup- 
pose we choose not t» sample the lot. Starting at node (a) at the left, therefore, 
and following the "do not inspect" branch of the tree, we move along to node (b) where 
we face the choice of either accepting or rejecting the lot. If we make the choice of 
rejecting the lot, we move tc node (c). There are now three possibilities; the lot is 
either 2 percent, 5 pertent or 7 percent defective. Weighting the ic 
ces of these three possible outcomes by their prior probabilities, 0.50, 0.25 and 0. 2s, 
we obtain a loss of $100 as the expected monetary value of this decision. This is in- 
dicated at node (c). A similar calculation for the decision to accept the lot yields 
a loss of $200, shown at node (d). Now the choice is between rejection or acceptance. 
The optimal decision is obviously to reject the lot incurring the smaller loss. The 
branch marked "accept" is blocked off, therefore, as indicated by the two wavy lines. 
Moving back to vertex (b), $100 is shown there, indicating the loss that would be ia- 
curred if we take the optimal act under the choice "do not inspect," 


On the other hand, suppose, at the outset, (at node (a)), our decision is to sam- 
ple and inspect n items. Making this choice, we move down to branch point (e). The 
results of the sampling inspection then determine which branch we follow. The pos- 
sible results of sampling nave been classified into three categories. The number of 
defectives, r, may have been equal to or less than some number dj (d, ¢n). It may 
have been greater than d) but equal to or less than do, (d;}< do <a). Finally, the 
number of defectives may have been greater than do. these three types of results, for 
purposes of brevity, are referred to as Type X, Type Y and Type Z information. In 
Table III, a joint frequency distribution is given for sample results and "states of 
nature" or "events". It may be assumed that these frequencies were derived from a 
large number of past observations, and therefore may be t°ken to represent probabil- 
ities in the relative frequency sense, For example, in the past, in .2 of the lots 
inspected, the number of defectives observed was dy or less and the lots were 2% de- 
fective. In terms of marginal frequencies, in .60 of the lots inspected Type X infor- 
mation was observed (r < d)), in .20 of the lots Type Y (d;}< r dp) was observed, 
and in .20 of the cases Type Z (r > dp) was observed, 


Returning now to the game tree, we observe the three branches representing I, Y, 
and Z types of information emanating from node (e), marked with their respective prob- 
abilities .60, .20, and .20, Suppose Type X information (r< d)) were observed. Mov- 
ing ahead to branch point (f), we can either accept or reject the lot. If we reject, 
we move to node (g); if we accept, to node (h), At these points, the probability 
questions that mst be answered are of the Bayes'rule or a posteriori variety. For 
example, the probabilities shown on the three branches stemming from (g), .70, .20, 
and .10 are the posterior probabilities, given Type X information, that the lots con- 
tain 2 percent, 5 percent and 7 percent defectives, respectively. Thus, they repre- 
sent revised probabilities of the occurrence of these three events, after having ob- 
served a particular type of sample.evidence. In Table IV are shown the calculation of 
these posterior probabilities by the use of Bayes' theorem, Table V shows these 
posterior probabilities for all three types of sample informtion. It may be noted 
that these probabilities may have been derived from Table III by dividing joint prob- 
abilities by the appropriate marginal probabilities, e.g. .70 = h2 , 


Now looking forward from node (g) and using the posterior probadilities .70, .20 and 
+10 as weights applied to the losses attached to the three events 2, 5 and 7 percent 
defective lots we obtain an expected payoff of a loss of $140. This figure is entered 
at node (g). Comparing it with the corresponding figure of $110 for "accept", we 
block off the action "reject" as being non-optimal. Therefore $110 is carried down 
to node (f), representing the payoff for the optimal act upon observing Type X rsa) 
information. Similar calculations yield $60 and $20 at (2) and (1) for Types Y 

Z information. Weighting these three payoffs by the prior probabilities of cteining 


(12) rere are, of course, other possible choices which are not considered in this il- 


lustration such as other types of sampling plans and 100 percent inspection. 


¥ 
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X, Y and Z information, we obtain a loss of $82 as the expected payoff of sampling and 
inspecting n items, Note that it is assumed that optimal acts will be taken after the 
observation of the sample evidence. Comparing the $82 with the $100 obtained under 
the act "do not inspect", we see that it would be worthwhile to pay up to $18 for the 
cost of sampling and inspection, This difference represented by the $18 is usually 
referred to 2s the "expected value of sample information". Since the cost of sampling 
and inspection is given as $5, it pays to sample and inspect mn articles, i.e., to pur- 
chase experimental evidence, rather than to act without obtaining this evidence, 


The point may be made here that decision theory analysis provides much more ra- 
tional answers to the questions, "should we sample? If so, how much should we be will- 
ing to pay for sample information?", than does the traditional statistical approach. 

It is in this area that decision theory makes a particularly valuable contribution - 
more so even than in answering the question as to which terminal act to take, once all 
the experimental evidence is in. 
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TABLE III 


JOINT FREQUENCY DISTRIBUTION OF SAMPLE 
RESULTS AND EVENTS 


I Sample Results 

lot Type X Type Type Z 

\_ Defective ad) < rz do r> do Total 

| 2 206 202 250 
5 +12 10 203 25 
7 215 225 

Total -60 -20 1,00 
TABLE IV 


CALCULATION OF POSTERIOR PROBABILITIES OF EVENTS 
GIVEN TYPE X INFORMATION (r < d)) 


Event 
lot ) 
Defective P (Ay) P P( Ay )P(X/A4) P(As/X) 
5 225 48 212 
7 225 206 210 


1,00 


TABLE V 


POSTERIOR PROBABILITIES OF EVENTS AFTER SAMPLE EVIDENCE 
OF TYPES X, Y AND Z HAS BEEN OBSERVED 


Event Sample Results 
(tet Percent Type X Type Y Type Z 
Defective 4) <r <dp 
2 .70 30 +10 
5 250 
? +10 .20 
1.00 1.00 1.00 


All possible decision rules or "strategies" will now be considered and utilized 
as a means of commenting on traditional hypothesis testing procedures, There are ten 
possible strategies that may have been considered in the illustration given above. 
These are enumerated in Table VI. An "R" denotes reject; an "A" denotes accept. 
Therefore, for example, decision rule or strategy number (1) means do not sample and 
always reject; decision rule () means sample and accept regardless of the type of 
sample information observed; decision rule (5) means sample and accept if Types X or 
Y information are observed, i.e. if do or fewer defectives are observed; decision 
rule (6) signifies acceptance if d; or fewer defectives are observed. A choice be- 
tween decision rules (5) and (6), therefore, would mean, in acceptance sampling terms, 

a choice between a single sampling plan with an acceptance number of do and one with 
acceptance number of dj). 
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TABLE VI 
POSSIBLE DECISION RULES BASED ON INFORMATION 
DERIVED FROM SINGLE SAMPLES OF SIZE a 


No Sampling Sampling 
Sample Information (2) __ (2) (20) 


Type I (r < R A R a a R R 
Type (d)< r< dp) R A R R A R A 
Type Z (r > do) R OA R A R R A A AR 


Operating characteristic curves may now be prepared for each of the decision rules. 
The only ones that are shown, however, are for strategies (5), (6) and an ideal © 
curve, Cigue 2). It can easily be shown that all other decision rules are "domin- 
ated" by (5) and (6); i.e. (5) and (6) are closer to the ideal curve for all fractions 
defective in incoming lots, Rules other than (5) and (6) under sampling do not, in 
fact, make much sense. For example, in rule (8), lots would be rejected if the num- 
ber of defectives in a sample of n articles were d) or less, but accepted otherwise. 
Although the points on the operating curve are joined by straight lines, probabilities 


should be read, of course, only for the discrete points 2 percent, 5 percent and 7 per- 
cent defective. 


Probability 0.74 
or 
Accepting 
the Lot 0.6 


i 


4 4Ideal 


7 


Percent Defective of Submitted Lot 


OC CURVES FOR STRATEGIES (5) AND (6) 
AND THE IDEAL OC CURVE 
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By means of the earlier analysis, decision rule (6) was shown to be optimal. 
That is, the conclusion on the basis of the game tree analysis was that the minimum 
opportunity loss would be obtained by accepting the lot if Type X (r< dj) infor- 
mation were observed and rejecting the lot if Type Y or Type Z information were 
observed (r > d)). 


Now, suppose this problem had been approached from the standpoint of a "null 
hypothesis" testing procedure, The hypothesis Hp: p = 2 percent would be tested at 
some preselected level of significance, Assume it had been decided that the maximum 
probability of a Type I error should be 5 percent. That is, it is decided that sub- 
mitted lots that are 2 percent defective should be rejected no more than 5 percent of 
the time. It will be noted from Figure 2 that rule (5) meets this criterion but mle 
(6) does not. Therefore, traditional hypothesis testing procedures would require the 
use of strategy (5), which has been shown to be non-optimal, Referring back to 
Figure 1, it can be seen that an expected loss of $11) would be incurred under 
strategy (5), ((.60 x $110) + (.20 x $220) + (.20 x $20)), as compared to an expected 
loss of only $82 under strategy (6). 


To summarize, Bayesian decision theory criticises traditional statistical hypoth- 
esis testing procedures on various grounds: (1) these hypothesis testing procedures 
do not provide a method for combining prior information with experimental evidence and 
(2) too much burden is placed on significance levels as a means of deciding between 
alternative acts - specifically, no formal method is provided for the inclusion of 
economic costs as a part of the decision making process. In conclusion, it should be . 
pointed out that Bayesian decision procedures are extremely flexible. For illustra- 
tive purposes, a single sampling scheme was investigated in this paper. As a general- 
ization, all possible sample sizes and acceptance number combinations could conceivably 
be tested to seek out optimal combinations. Furthermore, sensitivity analyses could 
be conducted to determine the effects of variations in prior probabilities and 
changes of costs in the payoff matrix, Finally, the procedures generalize to the 
consideration of more complicated sampling plans such as double sampling and 
sequential analysis. 
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WORK SAMPLING 


Wallace J. Richardson 
Professor of Industrial Engineering 
Lehigh University 


The purpose of this paper is to discuss the technique of Work Sampling, and to 
suggest ways in which the practicioner in the field of Statistical Quality Control may 
be of help to himself and his management with respect to the use of Work Sampling. Be- 
cause Work Sampling was originated by an eminent statistician, L.H.C. Tippett, and be- 
cause this audience is quite sophisticated in its acceptance of the concept of gather- 
ing information by means of sampling, no extended discussion of the technique is appro- 
priate. However, I should like to offer a definition of the technique: ‘ 


(1) “Work Sampling is a measurement technique for the quantitative analysis, 

in terms of time, of the activity of men, machines, or of any observable state 
or condition of operation. Work Sampling is particularly useful in the analysis 
of non-repetitive or irregularly occurring activity, where no complete methods 
and frequency distribution is available..... 


A Work Sampling study consists of a large number of observations taken at ran- 
dom intervals; in taking the observations, the state or condition of the object 
of study is noted, and this state is classified into predefined categories of 
activity pertenent to the particular work situation. From the proportions of 
observations in each category, inferences are drawn concerning the total work 
activity under study ..... 


The underlying theory of Work Sampling is that the percentage of observations 
recording a man or machine as idle, working, or in any other condition reflects 
to a known degree of accuracy the average percentage of time actually spent in 
that state or condition. If observations are randomly distributed over a suf- 
ficiently long period of time, this theory is held to be true, regardless of 
the nature of the observed activity." 


The important thing to remember in using Work Sampling is that it is merely one of 
several work measurement techniques. As in any measurement situation, the first thing 
to be done is to match the measuring tool to the demands of the job. The particular 
characteristic of directed human effort, or work, that is most descriptive of its na- 
ture is the extent to which particular motion patterns repeat themselves, or in other 
words whether the job represents "mass production" or "job shop" activity. Too many 
people equate work measurement with the stop watch, and thus ignore significant advances 
in the field which have taken place in the past few years. Everyone here has heard fac- 
tory management say "Quality Control won't work here, our lots are too small." These 
same people say “Work Measurement does not apply in our area, our work is too irregular." 
And in both cases the real answer is that techniques designed for measuring large num- 
bers of uniform events cannot be expected to work in situations where we do not have 
large numbers of uniform events. 


Thus, the first thing we must do is to develop a scale of precision of measurement 
which can be matched with the degree to which the work being measured is repetitive. 
All the work measurement techniques to be described are in use at present, but of course 
it must be recognized that to ask "Which is best?" is foolish. The question should be: 
“Which is best for the situation at hand?" For highly repetitive work, “hich does not 
involve a process-controlled element, the predetermined human work time is appropriate. 
In this technique, "basic" movements of the hands and body members are assigned times 
for their accomplishment. These times are in the order of fractions of a second, and 
a motion pattern is described in terms of specific movements with corresponding times. 
M™M, Work-Factor, BMT and other proprietary systems are typical of such predetermined 
work times. They are very useful, but expensive and time-consuming. One company esti- 
mates that it takes six hours' time to build up data on one minutes’ worth of work. How- 
ever, for repetitive work this technique is most effective. 


Stop watch time study is widely used on work that is repetitive, but in which pro- 
cess variations may occur. This technique has its faults, but is still widely used. 
Its reliability compares closely with predetermined work times, on the order of minus 
or plus ten percent at the two sigma level of confidence. But the work must be repet- 
itive, and skill is required in application. Essentially a sampling technique, stop 
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watch time study is susceptible of control-chart analysis, and variability of readings 
frequently is analyzed using standard Quality Control Chart factors. 


From both predetermined human work times and time study we develop tables of stan- 
dard data for use in rate setting. These tables relate some physical characteristic of 
the product to the amount of time required to perform a given operation on the product. 
In passing, this sounds familiar to the Quality Control man because this is almost ex- 
actly what he does in his work. It might be added that this is an area in which he can 
contribute to work measurement to a much greater extent than he now does. 


Moving now to work that is non-repetitive in nature, such as clerical, maintenance, 
material handling and research and development, we find that such activity has been lar- 
gely left unmeasured. This is true because prior to the use of Work Sampling, no meas- 
urement technique seemed appropriate, and also because until the past few years this 
work was not recognized as being capable of measurement and control. Better management 
practice, and the growing impact of automatic devices have changed this. Now the emph- 
asis is on equipment and personnel utilization, rather than on the “work content" of the 
job. Further, many labor groups, faced with technological unemployment, have resisted 
management's efforts toward cost reduction through layoff. The airline engineer's strike, 
the harbor pilot's strike and the last steel strike all had as the central issue the 
question of manning or work rules. It has therefore become more important to establish 
labor standards in non-repetitive work. For this Work Sampling has proved to be a most 
effective technique. Because Work Sampling is convenient and economical, it is quite 
reasonable to suggest that a study be taken to extend over a period of a month, or better 
still, five weeks. In this way, if there is a monthly cycle it can be defined; also, 
the advantages of a larger sample, spread over a longer period of time, need not be str- 
essed to this group. 


But Work Sampling in and of itself provides only information about the input to 
the job, in terms of time utilization of the people involved. This must be matched with 
an output in terms of fairly well-defined work units. In maintenance, for example, units 
of output might take the form of job orders, routine preventive inspections and hours of 
unscheduled down time. In this latter unit, of course, a low figure is usually consid- 
ered desireable. It is in this area of defining and measuring units of output that the 
Quality Control man can be of great assistance to the effort. This is true because he 
is accustomed to dealing in distributions, and the causes and definition of variability. 
In some cases, an old hand in time study will carry over from his "savvy" in measuring 
repetitive work the concept that unless every single work unit is very precisely describ- 
ed (and equally precisely measured), no true measurement exists. Unfortunately, in deal- 
ing with non-repetitive work, this simply is not true by the definition of "non-repeti- 
tive work." The secret of ultimate control of this type of labor cost is to close in on 
the exact figures by degrees, starting with an imprecise measurement, and exercising 
judgment as to the point at which it becomes uneconomical to pursue precision. The Qual- 
ity Control man is accustomed to doing this, because control limits are after only a set 
of decision rules, predetermined when the sampling plan is designed. The time study man 
often is used to working only with averages, and imparts to many of these a sort of sa- 
cred quality in contradiction of the facts. 


This is not to say that a competent, well trained Industrial Engineer harbors these 
notions. But unfortunately there are many in work measurement who think that "X-bar is 
the name of Hopalong Cassidey's ranch", and you may encounter some of these. There are 
a large number of well-trained Industrial Engineers, and I invite your attention to the 
"JOURNAL OF INDUSTRIAL ENGINEERING", which is a first-class publication of the American 
Institute of Industrial Mngineers. This is a professional society in the best sense of 
the word. Among members of this society there is a broad understanding of statistical 
theory. But there are many practicing in the field of work measurement who are not know- 
ledgeable in statistics. Further, Work Sampling, a simple and direct measurement, is 
quite often used by managers and others who claim no competence in work measurement. 
These people in particular are in need of your aid. 


Basically, the problem in selection of work units is the same one that Quality Con- 
trol engineers face every day. The units must be defined, an accurate count must be 
made of volumes or numbers, and above all the units must be meaningful in their relation- 
ship to the categories of the work sampling study. To take these one at a time, anyone 
here will recognize the problem of definition. Work units must be classified, a control 
point set up to obtain consistent sorting by classes, and a mechanism set up by which 
the integrity of the classification is maintained. For example, one maintenance depart- 
ment increased its volume of work under a control program by simply starting to record 
as "job orders" certain small repair jobs that previously had not been recorded, but 
were picked up by the mechanics in their spare time. In terms of work measurer it, work 
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unit counts are an essential, because we must ideatify hours with output, and looking 
ahead to eventual control and perhaps incentive payment, the system should be consistent 
with management's desire to obtain control. Notice that the word is "obtain." This is 
used because most applications of Work Sampling to work standards come where no control 
exists at the start of the study. Finally, the problem of relating the work unit to 

the categories of activity is important in the same sense that relating a physical char- 
acteristic of product to the performance of that product is important in Quality Control. 
All we have from a Work Sampling study is a series of category proportions, and we must 
then relate these to work units to produce a standard of "time per unit". 


Great emphasis has been placed on the work unit because it is in this area that 
many Work Sampling studies are lacking. Also it is in this area that greatest progress 
can be made toward the eventual reduction of cost. Generally speaking, we do not ex- 
pect craftsmen or clerical workers to work harder when they are working under a system 
of labor control. The most realistic objective is to have them devote more of their 
time to productive activity, and less to non-productive activity. One basic concept in 
which much progress can be made is the idea that maintenance, clerical and other “indir- 
ect" employees really perform a service, the quality and quantity of which can vary. 
This is consonant with the thinking of the individual who appreciates the concept of 
the distribution, but unfortunately it is not widely appreciated among managers of some 
activities. Make a test yourself. Ask your maintenance supervisor what his output was 
for a previous month. The chances are good that he will reply "X thousand dollars", or 
perhaps he will say "I had seventy mechanics on the payroll." The first step in get- 
ting a better answer, of course, is to define the output. Simply by taking a listing 
of jobs, and forcing yourself to write opposite each the output of these jobs in terms 
of work units of known quality you will take a giant step toward control of labor cost. 
This is true because one way to reduce cost is to eliminate work, and many work units 
will not stand close scrutiny when the criterion is the necessity of the task in the 
face of planned cost reduction. But unless you ask the hard question of necessity cost 
reductions will not come or. a rational basis. 


Once work units have been established and the Work Sampling results matched against 
these, another job must be done in which you can be of real assistance. That is in es- 
tablishing the standards themselves through correlation and regression techniques. The 
end result of a study should be a series of graphs or formulae by which a known output 
can be related to a standard input of man hcurs necessary for its accomplishment. If 
competent industrial engineers are working on the problem, your advice will be sought. 
If no such men are available, you may be called on to establish these relationships. 

A word of caution is in order here. Be sure to question the degree of accuracy which 
is desired. If the program is one designed to obtain the first step in control, an 
error of the order of perhaps fifteen percent may be’ normal. As more experience is 
gained, slightly more accurate results may be obtainable. 


One basic question on which you may be asked to give advice is the sample size in 
Work Sampling. Since Work Sampling studies have so many applications, a wide variety 
of answers may be given. The most common fault seems te be to insist upon a high de- 
gree of reliability for all studies, and the application of rigid rules to obtain this. 
As in Statistical Quality Control, all sampling plans do not have the same objective, 
and such blanket rules can be silly. The rule of reason should prevail, and the basic 
parameters of the difficulty in obtaining observations, the size of the group being ob- 
served, and the other demands made upon the observers' time should be considered. Ar- 
ticles have been written seriously suggestion sample sizes of over three million obser- 
vations. But since the most common use of Work Sampling is in fact-finding and in the 
establishment of standards for non-repetitive work, it is obvious that samples of a 
thousand or so observations on particular groups of employees seem more logical. 


One final thought should be advanced. The very acceptance by some managements of 
the concepts that sampling is in itself a useful tool of management is of importance to 
this audience. The further questions raised by the definition of work units and their 
correlation with the results of Work Sampling provide those with a skill in statistica? 
inference with a golden opportunity to extend their service to management. It is sug- 
gested that this opportunity should be seized. With the trend toward more mechanized 
operation, greater emphasis of equipment utilization, and the growing importance of ser- 
vice-type non-repetitive work, the technique of Work Sampling is bound to become more 
widely used. You in this audience can be of real assistance to your employers in pro- 
viding some.of the necessary statistical advice. You will also find that it is an in- 
teresting technique, and one that will provide an opportunity in its application for 
your own professional growth. 
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MANAGEMENT'S CHALLENGE FOR PRODUCT RELIABILITY 


W. A. MacCrehan 
Executive Asst. for Reliability 
Avionics - Bendix Radio 
The Bendix Corporation 
Kansas City, Mo. 


For many years the customer has accepted products that initially can be inspected 
and tested, and considered satisfactory at a "time zero" condition which upon 
occasion in field use have proven to have deficiencies preventing optimum customer oy 
useage. Examples of estremely high maintainability costs because of product failure to ot 
meet operational useage requirements have been cited again and again in the reports of 
James Bridges of OASD (R & E) - Commercial industry could offer an endless list of 
warranty, customer goodwill adjustments and other cost balancing options to "sooth" 
customer irritation, 


With recent years, an appreciable growth has occurred in contractual requirements 
for pre-test under field use enviornments that concern the time domain: e.g. "shall 
run 100 hours of use with 90% confidence" ... "at least 85 out of each 100 shall fire 
when required"..."Shelf life deterioration will not be greater than 5% over a two year 
period"..."Minimum life shall be 50 hours within 95% confidence level", 


With the impact of military specifications containing difinitive reliability 
clauses such as AGREE procedures and in the commercial field, with automobile manu- 
facturers offering 12 months warranty where previously three month warranty had pre- 
vailed, it is clear the trend for assurance of product performance is becoming a 
factual customer requirement, 


How does management prepare to meet this customer requirement?... What philosophy 
change must take place and how does one organize to insure control through the 
Engineering Design and on to the manufactured product. 


This presentation approaches the problem from the Top executive view of estab- 
lishing controls and balance to assure product reliability. The appraisal deals with 
delegated authority, responsibilities, audits and costs, 
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HOW RCA MEETS AGREE TEST REQUIREMENTS 
Paul Jay Goldin, Manager, Statistical Quality Control 
Airborne Sy Divisi © Def Electronic Products * Radio Corporation of Amorica 


RCA has been and is presently producing severai electronic systems which must meet the 
reliability test requirements of the Task Group 3 section of the Advisory Group on the Relia- 
bility of Electronic Equipment document (AGREE). The Task Group 3 section of AGREE has 
been adopted by the Air Force and is being included in revised form in most equipment con- 
tracts now being awarded. This paper will discuss the implementation and follow-up that was 
needed to meet the AGREE requirements. A comprehensive review of the results encountered 
during AGREE testing will also be provided. 


The Air Force adaptation of AGREE requires continuous reliability testing of hardware 
throughout the life of a contract to assure that equipments meet contractual mean-time-between 
failure (MTBF) requirements, and periodic longevity tests to assure that equipments meet the 
contractual service life requirements. To provide this assurance the contractor must place the 
first twenty-two equipments produced on a contract into a reliability test, each to be tested a 
specified minimum number of hours. This reliability test is conducted under a combination of 
environmental conditions which simulate the environment the equipments will face in service 
(figure 1). As the equipments are tested, operating time and failures are accumulated and 
matched against a sequential sampling plan having time and failures as the two variables (figure 
2). The contractual mean-time between failure requirement, plus the desired confidence of 
measurement as a tolerance around the MTBF, is the basis for the derivation of the sequen- 
tial plans' accept or reject criteria. The reliability test continues until either an accept or 
reject decision is reached. When an accept decision is reached, the initial month's ship- 
ments can be made. Each month thereafter, a specified sample of equipments is placed on 
reliability test and operated until a decision is reached. Shipments of equipments not sub- 
jected to the reliability test can be made only if an accept decision has been obtained. A 
reject decision halts shipments until corrective action is taken, and the contractor can verify 
the effectiveness of his corrective action by successfully reliability testing either 37 or 50 
consecutive equipments against the criteria of the sequential sampling plan. 


It should be apparent from this discussion of the AGREE reliability test that failure to 
pass the initial and monthly reliability tests can cause the contractor severe hardships. Sched- 
ules will be missed, billings will be lost, and costly rework and retest will result before ship- 
ments can again be resumed. The contractor, therefore, should take positive steps prior to 
the reliability test to assure himself that the finished product will meet the AGREE require- 
ments. 


PRELIMINARY PLANNING 


RCA received its first production equipment contract requiring AGREE in February, 1959, 
for Airborne Time Division Data Link systems, Although RCA had prided itself on producing 
equipments of high reliability, this was the first-time that RCA would be contractually required 
to demonstrate monthly compliance with a contractual MTBF under "AGREE" rules. 


As thought was given to meeting the AGREE requirements, it became apparent that RCA’s 
tight process control system would have to be further re-inforced, This process control system 
would have a three-fold objective; first: to push the debugging phase as far back in the assembly 
and test cycle as possible; second: to reduce the influences of the human factor; and third: to 
assure the customer and RCA that each system, whether it was subjected to the AGREE test or 
not, would meet the contractual MTBF. 


Table I compares the normal RCA Control of two years ago, and the controls now being 
used on all AGREE programs, An asterisk indicates that a new control has been established. 
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Table I — Comparison of Process Controls 


NORMAL PROGRAM 


AGREE PROGRAM 


10. 


11, 
12, 
13. 


14, 


15. 


16. 


17. 


18. 


Contractual Review 

Design Review 

Reliability Predictions 
Quality Control Acceptance of 
Engineering Models 


Award of Purchase Orders. 


Supplier Quality Field Coverage 


Purchased Material Inspection 


Printed Circuit Assembly and Test. 


Assembly and Wiring. 


Black Box Bench Tests. 


System Test 

System Burn-in 
Final Acceptance 
Packing and Shipping 


Lab. Analysis of Repetitive 
Failures 


Data Reporting 


Training Programs and Instruction, 
Testers and Test Supervision. 
Inspectors 

Assembly Operators. 


6. 


10. 


11. 
12, 
13. 
14, 


15. 


16. 


17. 


18. 


Contractual Review 
Design Review 
Reliability Predictions 


Quality Control Acceptance of 
Engineering Models 


Pre-purchase Order Award Surveys 
Vendor Ratings. 
Award of Purchase Orders 


Supplier Quality Field Coverage 
Suppliers Burn-in Black Boxes 
Purchase Specification Requires MTBF 
Demonstration 


Purchased Material Inspection 


Printed Circuit Assembly and Test 
Printed Circuit Burn-In 
Printed Circuit Environmental Tests 


Visual Acuity, Color Discrimination, and 
Manual Dexterity tests for operators 

New wiring, soldering techniques 
Controlled atmosphere for Assembly and 
Test 

Increased Programmed Automatic Circuit 
Checkers 


Black Box Bench Tests 

Black Box Burn-in Tests emploving vari- 
able combinations of environments, such 
as Temperature and Vibration 

System Test 

System Reliability Tests (AGREE) 

Final Acceptance 

Packing and Shipping 

Lab. Analysis of ALL Failures 

Return of Failed Parts to Suppliers for 
Corrective Action 

Data Reporting 

Special AGREE Forms and Tags. 
Engineering Failure Analysis. 

Training Programs and Instruction. 
Testers and Test Supervision. 


Inspectors 
Management 


PROCESS CONTROL 


Quality Control in Engineering 


Quality Control had been involved in the processing, inspection and acceptance of 
Engineering-built models for several years prior to AGREE. It was realized, however, that 
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many initial production problems could be eliminated if Quality and Process personnel were 
assigned to Engineering through the design and prototype stages. Quality Control Technicians 
and Process personnel were made responsible for determining the compatibility of mechanical 
designs with military specifications and assembly techniques, development of assembly and 
test procedures, and inspection of assemblies to detect and correct MIL-SPEC and good quality 
practice violations, in order to make the transition from engineering development to factory 
production as smooth as possible. As a result of this integrated program, the number of 
Change Notices was reduced materially. 


The success of these activities has led to establishing an Engineering Quality Control 
section which has the responsibility of servicing all military engineering contracts, regardless 
of whether or not AGREE is involved. 


Burn-in 


As an outgrowth of many meetings and discussions prior to the start of production, it was 
concluded that one of the two most important steps that manufacturing could take to assure equip- 
ment passing the AGREE test would be to debug the systems as far back in the production cycle 
as possible. Accordingly, exhaustive environmentally stressed debugging tests were estab- 
lished at selected components, printed circuit, selected sub-assemblies, and black box levels, 
as opposed to debugging tests under ambient bench test conditions on black boxes and systems. 


All suppliers who build major assemblies are required to burn-in their units for given 
periods of time under the same environmental conditions imposed on RCA built units. All 
printed boards are subjected to a dynamic bench test at room temperature; at the temperature 
extremes of the specifications; and at room temperature again. Under these conditions, each 
board receives about 5 hours of testing. Each black box is tested for 48 hours under the same 
environment as the system encounters in the AGREE test. This debugging procedure results in 
all black boxes, sub-assemblies, and components undergoing the same environmental stress 
whether they are sampled in AGREE or not. (The results of the complete debugging cycle are 
shown in figure 3.) 


Assembly 


Tightly controlling assembly processes and methods was considered by management to be 
just as important a technique in producing reliability as pushing the debugging phase back into 
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the manufacturing cycle. In addition to normal controls, such as variable and attribute control 
charts, failure reporting, and laboratory analysis of repetitive failures, special tests for as- 
sembly operators, a controlled assembly environment, new soldering methods, laboratory 
analysis of all failures, special failure reporting forms, and a closed information loop involv- 
ing Factory and Engineering Quality Control were introduced. 


All assembly operators are required to pass a visual acuity, color discrimination, and 
manual dexterity test, prior tobeing assignedtoan AGREE program. Operators are equipped 
with thermostat controlled soldering irons for normal soldering, and resistance soldering 
irons for soldering in restricted areas. 


Wiring and soldering techniques were subjected to int investigation to keep errors to a 
minimum. Where bly pr had called for soldering three wires to the same terminal 
in one operation, the process was revised to solder the upper, middle, and lower wires individ- 
ually. Although three operations are substituted for one, it was found that the incidence of no 
solders and rosin solders was greatly reduced. Specially designed cardboard frames that support 
cables and prevent them from being flexed during assembly and handling were introduced and 
have been very successful in preventing broken strands. Thermal stripping of wire is used in 
many areas to prevent nicking of wire strands and subsequent breakage. 


Failure Reporting and Analysis 


A major shortcoming of many quality programs is the lack of accurate failure reporting 
and analysis. Test personnel may treat failure reports, data recording, and tagging failed 
parts, as secondary tasks. This approach can not be tolerated where AGREE is concerned, 
since failure reporting and analysis provide the basis for evaluating process controls. 


The AGREE document spells out the type of forms that must be maintained during the re- 
liability and longevity tests, and the failed part analysis required. This information is proc- 
essed into six forms, the most important of which is the Failure Report (Figure 4). The 
Failure Report is a master which must be completed for every failure. The tester and his 
supervisor complete their section, and if 2 component or sub-assembly is involved, send the 
defective part and the failure master to the Quality Control Laboratory. The Quality Control 
Laboratory analyzes the part for mechanism of failure and reports its findings, Every failure 
report is then sent to Reliability and Design Engineering for review and recommendation, If 
the responsibility for failure is a supplier’s, the part and a failure report copy are sent to the 
supplier for his analysis and corrective action, If the responsibility is RCA’s, corrective 
action is taken by the responsible assembly or test area, or by Engineering. The entire failure 
reporting cycle is shown in figure 5. 


Although the concentration on failure reporting and analysis at the reliability test phase 
was needed from the standpoint of determining corrective action, it was actually after the fact. 
The information could be used to prove whether or not equipment met the MTBF requirement 
and to take action to prevent similiar failures the following months, but could not prevent the 


manufacturing activity from experiencing an AGREE reject decision should too many failures 
occur. 


At the initial planning stage, it was decided that to provide control and corrective action 
within each monthly AGREE demonstration, it would be necessary to analyze every part failure 
from printed circuit testing up to AGREE system testing. Although this was a drastic change 
from the normal procedure of analyzing only repetitive failures, it has played a major role in 
helping RCA to fulfill contractual MTBF requirements. 


The following are two exampies of what AGREE and prior to AGREE analysis can ac- 
complish. One of the first AGREE failures experienced was a film resistor which opened after 
a prolonged period of temperature cycling. Analysis indicated that the failure was due to a 
residual strain brought about by the packing density and the restriction placed on the parts by 
the printed board potting compound. This relationship became an excellent sorting mechanism 
for the more poorly made resistors. A combined effort by the supplier and RCA Engineering 
resulted in a new resistor design which eliminated the problem. The important point of this 
example and other similar ones is that without the pre-AGREE and AGREE environmental 
stressing, this condition may not have become known until failures occurred in the field, Then 
a costly and time-consuming retrofit program would have been necessary. By identifying the 
problem in the factory, corrective action could be made prior to shipment of systems, The 
above example, however, counted as a failure with respect to the AGREE accept-reject criteria. 
A second example illustrates how analysis prior to the AGREE test can prevent AGREE failures. 
Analysis of a wire wound resistor indicated that the leads were insecurely terminated in the 
rubber material of the resistor’s interior. This allowed the lead to rotate, breaking the con- 
tact internally. Based on this analysis, 2066 resistors in stock and on printed boards were 
X-rayed. The findings indicated that 50 resistors had the internal contact broken, The resis- 
tors were returned to our supplier for their corrective action as shown in figure 6, Similar 


failure analyses have resulted in many potential AGREE failures being weeded out prior to the 
AGREE test. 
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Figure 4. AGREE Failure Report 
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Figure 5. AGREE Failure Analysis 
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Figure 6. Example of Pre-AGREE Corrective Actions 


The above examples illustrate how analysis of an initial failure may prove extremely valu- 
able. Most important, however, is that a dual effort is required; first: AGREE test failure 
analysis to control month to month trends and to identify failures that would normally only show 
up in field use; second: prior to AGREE test failure analysis to indicate within month trends 
and provide corrective action prior to the AGREE test. 
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RESULTS OF AGREE TESTING 


The first RCA equipments to be subjected to AGREE were the Airborne Time Division 
Data Link systems. These equipments provide high speed communication of messages and con- 
trol information to manned and unmanned aircraft for intercept, tactical control, traffic con- 
trol, and other modes of operation. They are used in the century series interceptors and the 
Bomarc missile, and are designated as the ARR-60, ARR-61 and DRR-1. 


As of January 31, 1961, eighty-six ARR-61 systems and 77 ARR-60 systems 
had undergone the AGREE reliability test. Figures 7 and 8 depict the progress made 
by the ARR-60 and ARR-61 in meeting or exceeding contractual requirements (the DRR-1 curve 
is not shown as the equipment's MTBF has been 2) times greater than required since the start 
of production). Except for the ARR-60, the equipments started out meeting the contractual 
MTBF requirements. The ARR-60 was the first equipment to undergo the AGREE test, and 
experienced an AGREE Pre-Production reject decision due to a combination of design problems, 
plus the inexperience of assembly and test personnel with the product, It should be realized 
that the AGREE test is extremely responsive to problem areas, and one or two problems of a 
repetitive nature can very quickly cause a reject decision. Fortunately, the descriptive and 
laboratory ana’ ysis that was performed on each failure provided sufficient information so that 
corrective action could be implemented quickly. The lack of trouble with the ARR-61 and 
DRR-1 reflects the use of ARR-60 experience to correct potential trouble spots in those sys- 
tems prior to subjecting them to AGREE, 


Experience with AGREE has resulted in many reappraisals of our engineering and manu- 
facturing methods and procedures. Where there had been doubts about the effectiveness of 
AGREE, there is now confidence in AGREE as an effective means for improving reliability. It 
provides us with a method which is sensitive enough to quickly detect significant changes in our 
assembly prc » the prc of our component suppliers, or inadequate engineering. 


AGREE experience has enabled us to compress into about three months, the process and 
design changes that normally required considerable factory, Engineering type test, and field 
data before they could be initiated. Historically, this same corrective action on non-AGREE 
equipments has stretched over one or two years and longer. 


A comparison can be made relative to the complexity of several equipments produced in the 

and their factory achieved MTBF, versus the complexity of the equipments undergoing the 
AGREE test program, and their factory achieved MTBF and/or their contractual MTBF. This 
data is presented in Table I. Note that the AGREE equipments are in some cases two to five 
times as complex as the pre-AGREE equipments, yet have achieved or are required to achieve 
substantially higher MTBF's. This has been accomplished under environmental stress whereas 
the pre-AGREE equipments MTBF's were measured under an ambient environment. 


Table I! — Comparison of Pre-AGREE and AGREE Equipment MTBF 


PRE-AGREE TYPE OF PART | FACTORY | CONTRACTUAL 
EQUIPMENTS EQUIPMENT COUNT MTBF MTBF 
AN/ARC-21 UHF Airborne Transceiver 2000 200 NA 
AN/ARC-34 VHF Airborne Transceiver 1250 250 NA 
MA-10 Airborne Fire Control 900 50 NA 
MG-10 Airborne Fire Control 10, 000 20 NA 
AGREE 
EQUIPMENTS 
AN/ARR-60 Aircraft Time Division Data Link 5900 204 200 
AN/ARR-61 Aircraft Time Division Data Link 2200 363 250 
AN/DRR-1 Bomarc Time Division Data Link 4000 118 5 
TSM-22 Data Link Ground Test Equipment 5800 - 250 
TSM-29 Data Link Ground Test Equipment 4500 - 250° 
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Figure 8. AN/ARR-61 Cumulative MTBF vs Production Month — 1960 


OBSERVATIONS 


Based on the AGREE testing and analysis performed to date, several significant observa- 
tions have been made. 


1. Temperature is the major discriminatory factor, and the difference in MTBF between 
the same system subjected to a cycle of -54°C to +55°C, or to a cycle of 0°C to +55° 
C, can be as great as 4 or 3 to 1, The DRR-1 data link exhibited a MTBF of 118 hours 
under the former conditions and over 500 hours under the latter conditions. 


by 
‘ 
\ 


208 1961 ASQC CONVENTION TRANSACTIONS 

2, As a corollary to statement one, it is felt that the number of hours the unit is burned-in 
is not nearly as important as the number of temperature cycles to which the unit is 
subjected. 

3. Low level vibration (25-60 CPS) is not a major factor in debugging transistorized black 
boxes. Our data indicates that about four hours of vibration is sufficient to weed out 
any workmanship errors, loose hardware, and parts that may be present. 

4. It is most important that the AGREE environmental test cycle be compatible with the 
equipment design requirements. Disagreement between design and test criteria can 
cause severe hardship to the contractor. 

5. It is most important that suppliers be made aware of the importance of the AGREE 
program, The need for taking corrective action based on the analysis results of only 
a single failure must be emphasized, This is a tedious educational process, but one 
which the contractor must take upon himself in order to assure acceptable results. 

6. Table III indicates the predicted, allocated, and actual part failure rates during relia- 
bility testing. Note that coils and inductors and miscellaneous parts were the only 
categories failing at a higher rate than expected. 

Table Il — AN/ARR-60 Averaged Predicted, Allocated and 
Actual Failure Rates 
= PREDICTED | ALLOCATED | ACTUAL 
CATEGORIES POPULATION (%/1000 hrs.) | (%/1000 hrs.) | (%/1000 hrs.) 
Blowers and Motors 1 0.090 0.233 - 
Capacitors 1205 0.010 0.026 0.037 
Diodes 1381 0.019 0.049 0.032 
Connectors 77 0.231 0.595 - 
Coils and Inductors 105 0.020 0.052 0.42 
Filters 5 0.055 0.142 - 
Resistors 2455 0.027 0.070 - 
Relays 6 0.446 1,153 - 
Switches 14 0.520 1.345 - 
Transformers 19 0.052 0. 134 - 
Tubes 15 2.233 5.760 5.96 
Transistors 578 0.031 0.080 0.077 
Misc. F, Y, etc. 61 0.101 0. 261 0.728 
NOTE: One failure was due to an unknown cause and could not be assigned. One failure was 


due to workmanship and was not assigned. 


CONCLUSION 


AGREE is advantageous to the Contractor. It provides the Contractor with assurance 
that delivered equipments meet reliability and longevity requirements, resulting in improved 
Contractor-Customer relations, fewer delinquent contracts, and ultimately lower product 
cost. It gives the contractor the incentive to upgrade his Quality Control System, and provides 
the opportunity for subsequent advances in the state of the art. 
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RELIABILITY - A MANAGEMENT PROBLEM 


Thomas A. Budne 
Industrial Consultant 
Great Neck, New York 


Introduction 


Behind every case of unreliability in the performance of complex equipment, there 
is ea technical explanation. Yet, the ultimate responsibility for failure to achieve giv- 
en reliability requirements must be assigned to management. 


We have at our disposal, today, sufficiently mature philosophy, methods of approach, 
technology and statistical methodology to assure the successful achievement of feasible 
reliability objectives, starting with the design of the product. While some or perhaps 
all of these elements, as we know them, will go through further stages of evolution to 
become more efficient and more effective than they are today, we do kmow that what we 
have is sufficient. These elements are the ingredients of the so-called reliability 
programs. 


In many commercial product companies, the continued improvement of product design, 
of manufacturing techniques and processes and of procurement practices has been a 
matter of normal progress and evolution. In supporting such improvement efforts, manage- 
ment has properly maintained a cautious attitude toward the costs for such efforts. As 
a matter of record in specific companies, the results of such efforts have paid off 
handsomely, not only in product quality and reliability, but in very substantial reduc- 
tions in manufacturing costs. 


The most effective of these efforts have gone into a planned and orderly program 
of sifting out predominant design, process and workmanship deficiencies; of getting to 
the basic causes of these deficiencies; and of careful follow-up on the corrective ac- 
tion which was indicated. In-house information which is continuously generated on ‘pro- 
duct, process, and workmanship deficiencies, and feedback information from the customer 
and from the sales force have provided the basis for such efforts. 


We have all experienced the benefits of increasing reliability in commercial pro- 
ducts such as gasoline engines, electric motors, refrigerators, etc. which have had 
the advantages of evolutionary improvement, in customer conscious companies. Most of us 
have experienced too, the continued unreliability in commercial products which fail to 
recieve the benefite of evolutionary improvement. Products which go through radical 
re-design every year prove our case. 


The Problem 


The critical reliability problem arises when revolution, rather than evolution 

of product design and production processes become the way of life; when size and wieght 
restrictions are made radically severe; when performance requirements and use conditions 
are pushed to the frontiers of our technology and experience; and when specified relia- 
bility objectives cannot clearly be attained by logical and practicable extensions of 
existing experience. In general, when our knowledge and experience is limited, we must 
fill this void with some sort of discipline which will minimize the risks of failure in 
achieving desired objectives. This is where the formal reliability program enters the 
picture. 


A comprehensive, planned, organized, coordinated and properly manned and funded 
reliability program can provide utmost assurance that reliability objectives will be 
achieved under the extenuating circumstances described above. The costs of such pro- 
grams are not trivial. However, we are familiar with the fact that we can buy time and 
buy experience whenever we are willing to pay the price. The reliability program is the 
best thing we can buy to cover our deficiencies in time, knowledge, and experience, in 
this situation. In the long range, we expect to see the costs of such programs pay off 
in achieved objectives and we expect, too, that the costs of such programs will diminish 
as the growth and accumulation of our technical knowledge and experience reduces the 
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degree of supplementary formal reliability effort which is necessary. 


In general, management has shown reluctance to approve the kind of funds necessary 
to support a comprehensive reliability program. In cases in which the full significance 
of exotic product specifications was not really understood and in which the needed re- 
liability program was not funded, the ultimate cost of the unreliable product, to the 
customer, if not to the producer himself, has been many times greater than a full- 
blown reliability program might have been. The situation has changed somewhat recently 
in that the customer for high reliability complex equipment, primarily the military 
services, takes the position now that the required reliability, as specified in the 
contract, must be demonstrated. At the same time, however, these same customers are 
indicating their understanding of the reliability problem by their expressed expecta- 
tion to pay for the necessary supplementary reliability effort. 


While a complete dissertation on all the elements of a comprehensive reliability 
program is not the purpose of this paper, a brief review of some of the major elements 
will be useful. 


Historical Note 


Many definitive elements of a reliability effort were formalized in 1957 with the 
official publication of a rather comprehensive report by the Advisory Group on Relia- 
bility of Electronics Equipment (AGREE) under the Office of the Assistant Secretary of 
Defense (1). This report culminated about seven years of background studies and activi- 
ties by specialists in both industry and government. Altogether, the report formalizes 
a philosophy, a set of definitions, a group of technicaland statistical procedures, and 
@ number of organizational and management principles which are considered essential for 
the understanding and achievement of reliability objectives. A large number of 
military documents and specifications on reliability, recently issued, follow AGREE 
recommendations very closely. 


The emphasis to-date has been predominantly on electronics equipment for the 
simple veason that this is where the problems and effects of unreliability have been 
most severe. 


Engineering Responsibilities 


In view of the problems which are created by the "revolutions" in product specifi- 
cations and reliability requirements, it isn't surprising that a major burden of relia- 
bility is placed on the engineering activity. The demands and complexities of the en- 
gineering job in areas of limited experience have created the need for specific supple- 
mentary activities which have been accepted as key elements of a reliability effort. 
Some of these elements are: 


Specification reviews as a means of bringing specialized skills into the problem 
of establishing completely detailed mutual agreement between the customer and the 
design engineer on what the product must do, how it must do it, the specific conditions 
under which it must be done, the certainty that it will be done successfully and any 
other considerations which can have a bearing on product performance and evaluation. 
There can be no measure of reliability until all relevant specifications on the product 
have been completely spelled out. 


Design reviews as a means of building assurance that all the specifications have 
been properly translated into an optimum design and allowing specialists of every con- 
cievable kind to contribute his special skills, knowledge, and experience. The anticipa- 
tion of all possible modes of failure, characteristic to the design, is a job which re- 
quires the participation of people who collectively, at least, have experienced all 
possible modes of failure. 


Reliability predictions through systems analyses are the means by which estimates | 
of reliability achievement during desgn are made. Inaccurate as such estimates may be, 
they provide a relative measure of reliability progress; a means by which areas of 
reliability deficiency can be identified in the design; a means by which the possible 
advantages and effects of redundancy, derating of parts, and trade-offs among various 
requirements can be determined. These predictions are continuously improved as more 
factual data from hardware evaluation becomes available. 


Reliability apportionment to subsystems and components is essential to provide sub- 
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contracting organizations with specific reliability objectives. Because all components 
do not functionally contribute equally to overall system reliability and because 
reliability prediction techniques are only approximations initially and because there 
may be interface problems which cannot be fully anticipated, the apportionment job is 
a difficult one and awaits improvements in supporting techniques. 


These are just a few of the reliability disciplines which have been evolved to meet 
the problems arising in the design phase from the fermentation of very complex and 
exotic product requirements. The critical nature of the design responsibility for ulti- 
mate reliability is clear. An upper limit of reliability potential is determined by the 
engineering design and every effort must be made to optimize it. Statistically designed 
experiments, computer simulation techniques and other scientific techniques must be 
used to maximum effectiveness. 


While design changes to improve reliability can be are are made at later stages, 
the severe costs of making such changes during the production phase or even later as 
retrofits cen readily justify the time and costs which are necessary to obtain an opti- 
mum design initially. This does not take into consideration the severe maintenance 
costs which have actually been experienced in the use of sub-standard reliability 
equipment. The total costs of procurement plus maintenance and repair for low reliabil- 
ity equipment has been shown, by a military agency, to be many times the same total 
costs for high reliability equipment. 


Quality Control and Manufacturing Responsibility 


The need for the participation of quality control and manufacturing specialists 
arises during the design phase and increases as development activity is undertaken. 
Problems of producibility, tolerancing, measurability, vendor and subcontractor evalua- 
tions, hardware evaluation and qualification, test equipment specification and standard- 
ization, environmental testing, etc. arise during the design and development phases and 
require the cooperative efforts of engineering, quality control and manufacturing per- 
sonnel. Manufacturing problems and deficiencies, too, must be es and prevented 
before they actually arise. 


A 
All the skills, talents, and experience of quality engineering and manufacturing 

specialists must enter the high reliability situation. Established standards of manu- 
facturing acceptability may require radical revision to meet the newer reliability ob- 
jectives. During the production phase we hope to maintain the potential design reliabil- 
ity to the utmost. Paralleling the optimization of engineering design, preduction sys- 
tems must be optimized to minimize any reduction in the potential inherent reliability 
of the product. 


Techniques which have been helpful in improving production processes have evolved 
over the past years. However, such techniques alone are often insufficient to achieve 
the optimum porduction situation. An overallplan of section, taking the total situation 
into consideration (2) must be formulated. The limited talent which is available for 
the optimization effort must be used with maximum effectiveness in the areas of maximum 
potential return. We contribute very little to the optimum situation by reducing the 
variation on a plating process when the major overall manufacturing deficiency is work- 
manship. 


As a matter of fact, statistics do indicate that a large part of the reliability 
problem can be attributed to workmanship deficiencies. The manufacturing optimization 
effort must tkae into cognizance even psychological considerations which may have a 
bearing on the problem. One such consideration is that of placing maximum responsibili- 
ty for workmanship on the operator himself. There has been a strong tendency in most 
companies to place a large share of responsiblity for deficient product on quality con- 
trol to the extent that true responsibility has, in fact, been removed from the opera- 
tor. While work conditions, tools, jigs, etc. can influence workmanship and should be 
improved when necessary, there is no substitute for the direct pressure of responsibili- 
ty for doing the job right the first time. Such pressures can be’ applied in a straight- 
forward manner with very rewarding results, (3). 


In general, quality engineering and quality control effort must be sharpened and 
made more effective to meet the challanges of higher reliability. Some interesting cri- 
teria to be considered are: Every quality control dollar invested must show a measur- 
able profit. Every potentially profitable dollar must be invested. 
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Philosophical Considerations 


As would be true for any complex problem which has been given so much emphasis and 
attention, there have been a number of philosophies, some conflicting, on the best way 
to handle the reliability problem. 


One of the earliest, propounded by Lusser emphasized testing to destruction or to 
failure as the means by which a basic understanding of the mechanism of failure, of the 
basic underlying chemical and physical phenomena contributing to failure could be obtain- 
ed. Such basic understanding would necessarily lead to better materials, parts, designs, 
and higher reliability. But Lusser's philosophy failed to achieve predominance today. 
There is great merit in his thinking, but it concentrated so heavily on this single 
basic idea that his approach failed to fill the immediate need for a total reliability 
program would would give assistance to the design engineer throughout his drawing board 
activities and then carry through into development, production, evaluation, field use 
and complete the circle of feedback and corrective action. 


Another philosophy has placed major emphasis on the combination of redundancy and 
high maintainability as the answer to the reliability problem. By incorporating redun- 
dancy in potentially low reliability areas of the system and by designing for ease of 
replacement and maintenance, high reliability equipment can supposedly be produced with 
less effort and lower cost. The opponents to this approach question the hypothesis of 
lower costs, noting the heavy replacement and repair costs on the relatively frequent 
failures of low reliability components as well as the added problems of inventory, trans- 
portation and handling of increased numbers of spare parts, etc. Another argument 
suggests that such an approach reduces the pressures for scientific and technological 
progress which must inevitably provide the reliability breakthroughs of the future and 
much higher reliability at much lower cost. 


The predominant philosophy today is the one which is presented in the AGREE report. 
It provides all the elements which are needed for a total reliability program forming 
the closed loop between initial design and field failure feedback. It is apparantly 
logical and practicable. It has brought about excellent results. However, it has some 
debatable aspects too. It places heavy emphasis on reliability prediction which in turn 
is dependent on a knowledge of the failure rates of parts under various conditions of 
application. In recognition of the parts problem the AGREE report recommended and was 
followed by a special study on this problem (4). A very heavy expenditure of time, ef- 
fort and test equipment must be made in the measuring of failure rates of parts. It can 
be argued that if a reasonable fraction of this parts testing effort would be devoted 
to statistically engineered investigations, designed to optimize and improve part de- 
signs and manufacturing processes, then failure rates could be reduced and the ground- 
work laid for greater reliability progress in the future. 


: It is likely that some combination of the many philosophies will evolve in the near 
future with enough variations to cover a large variety of basically diferent problems. 


Conclusion 


It can be seen that the overall effort for a comprehensive reliability program is 
tremendous even though we haven't discussed many important aspects such as: parts evalu- 
ation and procurement, coordination of subcontractor reliability efforts, failure report- 
ing systems, failure data analysis, failed parts analyses, reliability education and 
others. Above gll, the coordination of all the activities and functions which relate to 
the reliability effort can make the difference between a successful program and an un- 
successful one. 


Many difficult issues must be resolved such as: 


The determination of the most effective and appropriate philosophy for the company 
and its future. 


The most effective delegation of responsibility for the reliability effort in its 
various phases. 


The optimum allocation of funds which are both necessary and sufficient for both 
the immediate objectives and the long range objectives of the company. 


These and other issues must be resolved by management. In considering the total 
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picture it becomes obvious that the struggle for high reliability is at least as great 
a challenge to management skills as it is to technical skills. The days of evolutionary 
progress and development can be properly remembered with nostalgia. 
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RELIABILITY REQUIREMENTS IN PARTS SPECIFICATIONS 


Charles J. Brzezinski 
Office of the Assistant Secretary of Defense 
(Installations and Logistics) 


SUMMARY 
Reliability is a quality characteristic that defies definition and, for this 
reason, is a requirement that is difficult to describe under any circumstances. It is 
even more difficult to describe the reliability requirement in a specification for the 
kind of component part that is commonly referred to as the “hardware” which collective- 
ly comprises the building blocks of equipment. 


This paper discusses the problem of developing a procedure for incorporating reli- 
ability assurance provisions into specifications for component parts. Two major prob- 
lem areas are examined: (1) the problem of developing a testing program that not only 
is capable of identifying the reliability level which a part can be expected to meet 
but also is capable of assuring conformance to this level when so specified by the pur- 
chaser; and (2) the problem of fitting such a flexible testing program into the inher- 
ently inflexible provisions of a part specification. 


INTRODUCTION 

The purpose of this paper is to discuss the problem of developing a procedure for 
incorporating reliability assurance provisions into specifications for those many com- 
ponent parts, cammonly referred to as "hardware," which collectively comprise the basic 
building blocks of any major item of equipment. During the past two or three years a 
great amount of effort has been expended on the problem of developing a reliability 
assurance provision that might be readily incorporated into the many existing part 
specifications. Of the many methods proposed for specifying reliability, however, only 
one offers a solution both to the problem of the very large sample sizes required for 
life testing and to the problem of applicability to specifications for a wide variety 
of products. 


The reliability assurance provision discussed in this paper might be described as 
a multilevel reliability requirement inasmuch as it proposes to establish more than one 
level of quality and reliability in each specification. The primary purpose of this 
paper is to examine two problems related to the development of such a multilevel reli- 
ability requirement: (1) the problem of developing a life testing procedure which 
calls for reasonably small sample sizes and still is capable of not only disclosing 
the reliability level a part can be expected to meet but also assuring conformance to 
this level when so specified by the purchaser; and (2) the problem of fitting such a 
Plexible testing procedure into the inherently inflexible quality assurance provisions 
of a component part specification. 


THE MULTILEVEL RELIABILITY REQUIREMENT 

The basis for the miltilevel reliability requirement is the fact that component 
parts manufactured to existing specifications are often much more reliable than these 
specifications require them to be. It is impossible, however, to identify these reli- 
able parts by using the test procedures called for in existing specifications since 
these procedures can do no more than distinguish between product that exceeds minimm 
performance requirements and product that does not. The only way to identify those 
lots which are significantly more reliable than required by the specification is to re- 
test each lot using a larger sample than called for by the specification. 


This additional testing would not be necessary, of course, if the specification 
provided some means by which a purchaser would have the choice of a number of different 
sample sizes. Each purchaser could then choose the size needed to assure the level of 
reliability he desired. 
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National Symposium on Reliability and Quality Control and was subsequently incorpor- 
ated into the quality assurance provisions of a general specification for semiconductor 
devides, MIL-S-19500B. This first version of the mltilevel reliability requirement 
appears in MIL-S-19500B in the following form: 


ASQC LCS Code 815:)0:000 


ae 

215 
\ 


216 1961 ASQC CONVENTION TRANSACTIONS 


40.7.2.1 Table IV shows the minimm sample sizes necessary to assure 
@ maximum failure rate in percent per thousand hours, in the lot represented, 
with a confidence level of 90 percent for various failure acceptance numbers. 


Table IV - Minimm size of sample to be tested for 
1,000 hours to assure a failure rate no greater 
than the maximm failure rate with 90% confidence. 


Maximum 

Failure | 50/20] 10] 5 | 2 1 0.5 0.2 0.1 
Rate 

Accept. 

aioe Minimum Sample Sizes 

0 ah] 47/116} 231] 461] 1,152] 2,303 
1 9}20] 40] 79}195| 390] 778] 1,946] 3,892 
2 12| 28 | 55 |109|266] 533/1,065 | 2,662] 5,323 
3 15 | 35 | 69 | 137 | 333 1,337 | 3,341] 6,681 
4 19] 42) 83 398 | 798/1,599 | 3,997] 7, 
5 22/49 | 97 | 190 | 462 1,855 | 4,638 | 9,275 
6 25 | 56 | 110 | 217 | 528 | 1,054 | 2,107 | 5,267 | 10,533 
7 28 | 63 | 123 | 243 | 589 | 1,178 | 2,355 | 5, 11,771 
8 31 | 70 | 136 | 269 | 648 | 1,300 | 2,599 | 6,489 | 12,995 
9 34 | 76 | 149 | 294 | 709 | 1,421 | 2, 7,103 | 14,206 
10 37 | 83 | 161 | 319 | 770 | 1,542 | 3,082 | 7,704 | 15.97 
20 o> | w= | en | --- | ~-- | 2,705 | 5,410 | 13,525 | 27,050 
ho -- | -- | --- | --- | --- | 4, 9,880 | 24,700 | 49,400 


The above described version of the multilevel reliability requirement undoubtedly 
improved the original version of MIL-S-19500 but is obviously impractical. First, the 
sample sizes called for in the above table are much too large for sampling component 
parts that quite often are purchased in smaller lots than the sample sizes called for 
in the table, although it should be pointed out that the only alternative to such large 
sample sizes is less reliability assurance. 


A second inadequacy in this type of reliability requirement is that there is no 
provision for ascertaining whether the product to be tested is designed to meet the 
specified failure rate level. In effect, this version of the multilevel reliability 
requirement proposes to “test reliability into the product." 


New Concept Details 
The second phase in —_— of the multilevel reliability requirement is 
2 


illustrated in the report of the Ad Hoc Study Group on Part Specifications Manage- 
ment for Reliability. The Study Group proposal, while it did not solve the basic prob- 
lem, included two new concept details related to certain essential elements of the 
multilevel reliability requirement. One of these details was a proposal to "certify" 
each supplier as an approved source of supply for electronic parts according to the 
average level of reliability exhibited by his product. The second detail was a propos- 
al to tie in retention of "certification" at a failure rate level with "acceptance" 
life testing as a means for reducing sample sizes. These two concept details are re- 
flected in a "Reliability Level Qualification" requirement and a sequential sampling 
procedure developed for the improved multilevel reliability requirement examined in 
this paper. 


The third phase in the development of the mltilevel reliability requirement was 
initiated when the Ad Hoc Study Group discontinued the second phase in order to prepare 
their report. It was apparent that further study was needed to complete the require- 
ment and a manual of instructions would be needed as a guide for revising existing 
specifications. This task was undertaken immediately by members of the staff of the 
Quality Control and Reliability Division, Office of the Assistant Secretary of Defense 
(Installations and Logistics), who had participated in the work of the Study Group. 

At the time this paper went to print an interdepartmental task group was working on a 
draft of a proposed manual of instructions illustrated with a prototype specification 
embodying a multilevel reliability requirement based on the following concepts: 
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(1) A provision establishing a series of reliability levels, the lowest level 
based on the performance characteristics of product in current production. 


(2) A provision for "acceptance" life testing which assures at the 90% probabil- 
ity level that those lots passing the test either meet or exceed the reliability level 
for which the product is qualified. 


(3) A provision for the systematic accumulation and evaluation of "acceptance" 
life test data to provide an estimate of the reliability level actually attained as a 
basis for "qualifying" each supplier's product according to the level of reliability 
it can be expected to meet. 


(4) A provision for adjusting environmental requirements according to the relia- 
bility level for which a product is qualified. 


(5) A provision for the use of a modified sequential sampling plan at the higher 
reliability levels to reduce "acceptance" life test sample sizes to a practi-ul minimm. 


SPECIFICATION REQUIREMENTS 
Before proceeding with this discussion, it might be well to examine briefly same 
of the restrictions that the "hardware" type of component part imposes on the specifi- 
cation requirements: 


(1) ‘The part may be manufactured by many suppliers, each producing the same part 
insofar as size, shape, and minimum performance are concerned but a very different part 
in terms of material, design, construction detail, and mseximm performance. The speci- 
fication must assure interchangeability, quality and reliability without imposing re- 
strictions either on the method of manufacture or any action necessary for product 
improvement. 


(2) The part may be used in a wide variety of equipments, under various combina- 
tions of environmental stress, under continuous or intermittent operating conditions, 
and under full or only a partial load. The specification not only mst provide assur- 
ance that the part will perform its intended function satisfactorily under any of these 
conditions but also must provide this assurance with one standard testing procedure. 


(3) The part may be manufactured either under mass production conditions or in 
job lots and may be purchased in either small or large lots. The specification mst 
provide a sampling procedure that assures conformance equally well under any of these 
conditions. 


(4) The same part may be produced in many variations that are not interchangeable 
with each other but not necessarily different with respect to those design details that 
might affect reliability or capability to withstand environmental stresses. In order 
to reduce performance testing to a practical minimum, the specification mst classify 
these variations into appropriate family groups such that the testing of any member of 
the family can be used to evaluate the performance of the family as a whole. 


With the above considerations in mind let us now proceed to examine the multilevel 
reliability requirement as it appears in a revised specification for a typical compo- 
nent part. 


Revised Version of MIL-C-25C 

The specification selected to illustrate the multilevel reliability requirement 
in this paper is a general specification for capacitors, MIL-C-25C, which is being used 
as an illustration in the previously referred to manual of instructions for revising 
specifications. This specification covers several hundred variations of a paper or 
paper-plastic capacitor, hermetically sealed in a metallic case. The product it covers 
is just about the most typical piece of “hardware” that might have been chosen to prove 
the practicability of the multilevel reliability requirement. As previously explained, 
the specification should permit each supplier to produce the best capacitor that he can 
and should not restrict the choice of material, design and construction detail any more 
than necessary. Actually, the original version of MIL-C-25C imposed very few restric- 
tions on such details and required relatively little revision in this respect. How- 
ever, the basic principle is clearly expressed in the following requirement: 
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3.3 Material, Design and Construction Detail. This specification 
prescribes only those material, design and construction details required 
to assure interchangeability between units of product. All other such 
details shall be selected by the supplier and shall be suitable for the 
intended purpose. 


Qualification Requirements 

From the standpoint of reliability, the above requirement is important because it 
removes any restriction that might prevent suppliers from making design changes to in- 
prove product performance. But, in removing these restrictions, the requirement opens 
the door to design changes that may adversely affect performance. In addition, the re- 
quirement may adversely affect procurement activities because deliveries on a contract 
may be delayed whenever a supplier has trouble selecting suitable design detail. Thus, 
the above requirement must be supported by two additional specification provisions. 
One provision should give each purchaser same means for determining before a contract is 
placed that difficulties related to the selection of unspecified design detail will not 
cause unusual delivery delays. The second provision should provide protection against 
the adverse effect of design changes. These two provisions have been included in the 
revised version in the form of the following "qualification" requirements: 


3.2 Qualification. At the option of the procuring activity, 
awards will be made for only those products that, prior to the bid 
opening date, have been tested and qualified for inclusion;\in the appli- 
cable Qualified Products List. 


3.2.1 Initial Qualification. Initial qualification shall be re- 
quired for product required to meet the minimum quality and reliability 
level established by this specification. 


3.2.2 Periodic Qualification Evaluation. Product to be furnished 
under this specification shall be required > pass a periodic qualifica- 
tion evaluation test to evaluate the abilit, of units from current pro- 
duction to withstand the environmental stresses applicable to the relia- 
bility level for which the product is qualified. 


3.2.3 Reliability Level Qualification. Product to be furnished 
under this specification shall be required to prove that its potential 
reliability is equal to or higher than the reliability level it is 
expected to meet. 


Classification of Inspections 
Inasmuch as the above qualification requirements are intended to evaluate the suit- 


ability of material, design and construction details, it is important that these require- 
ments consist of tests specifically designed to accomplish the desired evaluation. Some 
tests in a specification are intended to check the suitability of design but others are 
not. It is possible to distinguish between them by classifying the examinations and 
tests in a specification into the following three functional groups. 


(1) Group 1, Conformance Inspections - These inspections are performed to ascer- 
tain that product conforms to standards established by the design. Each examination or 
test in this group involves a measurement or comparison to a design standard. 


(2) Group 2, Environmental Tests - These tests are performed to ascertain whether 
product conforming to design standards (per Group 1 Benes geome withstand certain envi- 
ronmental conditions to which it may be subjected in operation. These tests include 
environmental conditioning (e.g., subjection to shock, vibration, temperature variation, 
etc.) during or after which the Group 1 inspections are repeated to detect any change 
in the product. 


(3) Group 3, Performance Tests - These tests are intended to ascertain whether 
product conforming to design standards is capable of functioning satisfactorily through- 
out its projected life. These tests are characterized by a specified number of hours or 
cycles of functional operation during or after which certain Group 1 inspections are re- 
peated to detect any changes in the product. 


It is readily apparent that the examinations and tests in only two of the above de- 
scribed functional groups are capable of detecting design deficiencies. Group 1, Con- 
formance Inspections can be used to determine whether the product conforms to a design 
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standard but cannot be used to determine whether the standard itself is suitable. Thus, 
the examinations and tests comprising the qualification requirements should consist of 
the Group 2, Environmental Tests and Group 3, Performance Tests. 


The following is an example of the above described classification when applied to 
the examinations and tests in specification MIL-C-25C: 


4.1.1 Classification of Inspections. The examinations and tests 
that collectively comprise the quality and reliability assurance pro- 
visions of this specification are classified as shown in Table IV. 


Table IV Classification of Inspections 


Subgroup 1: 
Dimensional and visual, internal 
Material 
Dimensional and visual, external 
Subgroup 2: 
Dielectric withstanding voltage 
Insulation resistance 
Capacitance 
Seal 


Quality Factor 


Environmental Tests, Group II 
Subgroup 1: 
Berometric Pressure 
Vibration 


Moisture resistance 
Shock 


Performance Tests, Group III 
Capacitance change with temperature 
Life 


When examinations and tests having a common purpose are grouped as shown in the 
above specification requirement, it is possible to specify inspection procedures that 
assure each group will perform its intended function. The following principles indi- 
cate the type of procedure that should be specified: 


(1) Conformance inspections should be planned to detect defects which may be ac- 
quired at any time during the manufacturing process. 


(2) Environmental tests should be planned to detect design errors. These errors 
may be generated before production begins and at any time a design change is made. 


(3) Performance tests are intended to detect limited performance potential caused 
by certain combinations of normal variations in design and construction detail. Unde- 
sirable combinations of these variations may be generated either by design error or by 
the manufacturing process, therefore, the performance tests should be planned in the 
Same manner as conformance inspections. 


It should be apparent from the preceding discussion that the new reliability as- 
surance provision involves no new test methods or inspection procedures. However, it 
is important that all elements of the existing quality assurance provision be closely 
coordinated into an effective testing program,for each element has a direct effect on 
the amount of life testing required. For this reason the several elements of the qual- 
ity and reliability assurance provision have been classified as shown in the following 
requirement in the revised version of MIL-C-25C: 


f 
Conformance Inspections, Group 1 | 
Salt Spray 
Temperature Cycling 
Immersion Cycling 
Subgroup 2: 
Terminal strength 
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4.1.2 Classification of assurance provisions. The quality and 
reliability assurance provisions are classified as follows: 


Quality Conformance Inspection. See 4.3 

Reliability Conformance Testing. See 4.4 
a) Periodic Qualification Evaluation. See 4.5 
$3 Reliability Level Qualification. See 4.6 


fs Initial Qualification. See 4.2 
b 
c 


Test Methods. See 4.7. 


Initial Qualification 


Initial qualification requirements are based either on existing specification re- 
quirements or on characteristics of product in current production. For this reason, 
the initial qualification requirement may establish a minimum performance level well 
below the current state of the art. However, it is not important that the minimum per- 
formance level may be too low inasmuch as the multilevel reliability requirement pro- 
vides for additional higher levels. Product that exceeds this minimum level will, of 
course, be identified according to the level it can be expected to meet. The following 
example is the Initial Qualification requirement in the revised version of MIL-C-25C: 


4.2 Initial Qualification. 


4.2.1 Purpose. The purpose of initial qualification is to assure 
that material, design and construction detail selected by the supplier 
is capable of withstanding the minimum environmental requirements of the 
specification. 


4.2.2 Granting of initial qualification. Initial qualification 
shall be granted on the basis of objective evidence that the product has 
been subjected to and has passed the tests specified for initial quali- 
fication. A test report, certified by a responsible official of the sup- 
plier, may be considered as such objective evidence. 


4.2.3 Combined qualification. Initial qualification shall be granted 
for all sizes within a style if the largest size in the style passes the 
initial qualification tests. 


4.2.4 Initial qualification sample. The sample size for initial 
qualification shall be in accordance with Table 5. All units in the 
sample shall be the same item number. 


4.2.5 Initial qualification tests. Capacitors shall be subjected 
to the tests specified in Table 5 in the order shown. 


Table 5. Initial fication Tests 


Test Sample Failures 
Size Allowed 


|_ Flashpoint 

Environmental Tests 

Subgroup 1 
Barometric Pressure 
Vibration 6 1 


Moisture resistance 
Shock 
Performance Tests 


Capacitance change 
Life 12 1 


Subgroup 
Terminal strength 
} 6 } 1Ppl 


\ 
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alt Spray 
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Reliability Level Identification 

When a multilevel reliability requirement is incorporated into a specification, 
some provision must be made for marking parts to show the reliability level they can 
meet. The method for accomplishing this identification is illustreted in the following 
example of a revised specification requirement: 


1.2 Item mumber. Each capacitor covered by this specification is 
identified by an item number in the following form: 


ce N 27 I 


Type Style Reliability Capacitance Capacitance 
Level Tolerance 


1.2.1 Type. The code letters CP identify one type of direct current, 
paper or paper-plastic dielectric, fixed capacitors hermetically sealed in 
metallic cases. 


1.2.2 Style. The style is identified by the number of the detail 
specification. Capacitors of the same style are alike with respect to 
voltage rating, circuit, configuration, capacitance change, terminal, 
and insulation resistance. 


1.2.3 Reliability levels. The reliability level is identified by 
a@ single letter in accordance with Table 1. The failure rate shown for 
each level is based on two assumptions: (1) the distribution of time to 
failure is exponential, and (2) the life acceleration factor is 5 for 
tests at 140% of rated voltage and does not differ between the products 
of different suppliers. 


Table 1. Reliability Levels 


Code Letter Failure Rate in % 
per 1000 hours 
M 31% 
N 6% 
1.5 % 
P AG 
Q 1% 


Although numerical values for failure rate are shown in Table 1, it should be point- 
ed out that these values are only estimates based on assumptions and are shown for in- 
formation purposes only. The values shown in the table were computed in the following 
manner: 


Problem: Compute the failure rate in % per 1000 hours which will 
be rejected 9 out of 10 times by the sampling plan for 
the life test in the Initial Qualification requirement. 


Given: Sample size, n= 12 Acceptance number, c = 1 
Length of test, t = 250 hours 


Assume: Distribution of time to failure : Exponential 
Acceleration factor : § 


Solution: Let the failure rate = R 
let % failures at time t = P) 


From tables of Poisson's Exponential Binomial limit: 
When c = 1, there is a 90% probability of rejecting p 
when np, = 3.9 1 
Since n = 12, p, = 3.9/12 = 0.325 
and e 1000 = 1 - p; = 0.675 
From Tables of the exponential function: 


when e~ 3586 = 0.675, = 0.393 
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Solving for R, R=0. 
or R= per 1000 hours. 


Conformance ions 
Conformance insyec yeare performed to ascertain that product conforms to stand- 
ards established by the design. Imasmuch as practically all material, design and con- 
struction details are selected by the supplier, conformance inspections can be speci- 
fied in general serms only. The following is an example of the revised conformance 
inspection requirement: 


4.3 Quality Conformance Inspection. Conformance to quality require- 
ments shall be assured by subjecting the product to the examinations and 
tests comprising the Conformance Inspections, Group 1, of Table 4. 


4.3.1 Point of inspection. Conformance inspections shall be per- 
formed at whatever point in the manufacturing process is most suitable. 


4.3.2 Conformance inspection sampling. If conformance inspections 
are performed on a sampling basis, the sampling procedures used shall be 
consistent with the sampling requirements specified for reliability con- 
formance. (See 4.4). 


Reliability Conformance Tests 
It is not intent of this paper to propose a definition for reliability wut it 


is necessary to define the term "reliability level" as used herein. The reliability 
level-is a quantitative requirement for reliability regardless of the fact that it can- 
not be expressed in terms of a numerical value. Whether the failure rate for levels M 
and N are 31% and 6% is immaterial provided level N is lower than level M. Whatever 
the true failure rate for each of these levels, the following statement can be made re- 
garding the difference between these two levels and also any other pair of adjacent 
levels: 


"If the reliability level cf a lot is such that an 
M level sample will not pass the life test only 5 times out 
of 100, an N level sample from the lot will not pass the 
life test 90 times out of 100." 


The reliability conformance element of the quality assurance provision is designed 
to assure conformance of individual lots to the level of reliability for which a part 
has been qualified. As indicated in the following example of the reliability conform- 
ance requirement, a sample is tested from each "inspection lot." 


4.4 Reliability conformance. Conformance to reliability require- 
ments shall be assured by subjecting the product to the life test element 
of The Performance Tests, Group III, of Table 4. 


4.4.1 Life test sample. A sample shall be selected at random fron 
each inspection lot which has passed conformance inspection requirements. 
Samples shall be selected in accordance with Table 6. ‘The table lists 
five sample sizes and corresponding acceptance numbers for each relia- 
bility level. Choice of sample size is optional ded (1) the sample 
size is selected before the life test begins and (2) the sample size 
selected is one of the five sizes listed for the reliability level the 
product is expected to meet. 


Table 6. Life Test Sample Sizes 


0, P and 


(See 4.4.3) 


ia 


Acceptance Sample Sizes Applicable to Reliability Level 
M 
7 
12 
16 
21 
25 


| 
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4.4.2 Inspection lot. The life test sample shall be selected 
at random from an inspection lot consisting of capacitors of the same 
style, manufactured as a continuous production run. Inspection lots 
may also be formed by combining * more production runs provided 
that all capacitors in the lot ar: « same item number. 


The sampling plans specified for Levels M and N are Lot Tolerance Percent Defective 
(LIPD) type sampling plans derived in the following manner: 


Step 1, Sampling plan for Level M 


Given: n = 12 and c = 1 from Table IX of MIL-C-25C 
and t = 250 from 4.6.14 of MIL-C-25C 
Assumed acceleration factor = 5:1 at 140% of rated voltage. 


From tables of Poisson's Exponential Binomial Limit: 


np, = 3-9 when P, = .10 andc =1 


P, 39/12 = «325 when n = 12 
Using Pp, = .325 and values for np, from the Poisson tables: 


c np, np,/P, n 
2.3 2325 Tol 7 
1 3.9 325 12.0 12 
2 5.3 2325 16.3 16 
3 6.7 2325 20.6 21 


With this type of sampling plan, each of the five sample sizes listed and its cor- 
responding acceptance number will reject 90% of the time the same reliability level that 
would be rejected 90% of the time by the n = 12, c = 1 sampling plan in the existing 
specification. 


Ss 2 for Levels N, 0, P and 
By a fortunate coincidence, a minimum amount of life test 
data is required to distinguish between adjacent reliability levels when 
P for the largest sample size at the lower level is made p, for the next 
r level. 
From the Poisson tables: When c = 4, =P, 1.97 


The largest n at the M level = 25 


0, P and Q levels. The sample sizes for these four levels are show in 
the following table. 


le Sizes ted per 1 and 2 
Acceptance |Sample sizes corresponding to Reliability Level 
Number 

M N P Q 

te) 7 29 121 478 19ho 

1 12 4g 200 812 3290 

2 16 67 272 1010 4k70 

3 21 85 343 1390 5650 

4 25 101 Log 1660 6740 


| 
H 
Therefore 1.97/25 = .0788 = P, » level. 
Step 1 can now be repeated to compute the same sizes for the ? 
N level and the procedure repeated to compute the sample sizes for the 
| 
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The above tabulation of sample sizes illustrates the rapid increase in sample 
sizes from one level to the next. At some point, such as level O in the above tabula- 
tion, the sample sizes computed in accerdance with Steps 1 and 2 become impractically 
large 


Sequential Sampling Procedures 


The next step, therefore, is to specify a different type of sampling procedure for 
the 0, P and Q levels. Depending on the product, it may be possible to lengthen the 
life test or to make the test conditions more stringent, which would permit the use of 
smaller sample sizes at the O and possibly P levels. But, as shown by the above tabu- 
lation, the sample sizes increase so rapidly from one level to the next that even this 
alternative is not completely satisfactory. 


The only recourse, then, is to develop a plan which may not provide as much pro- 
tection as the plans computed in Steps 1 and 2 but approache~ this level of protection 
without increasing sample sizes. This objective can be attained by the following pro- 
cedure: 


(1) Use the N level sampling plan to "reject" lots. This assures a 90% probabil- 
ity that no lots will be considered for acceptance at the 0, P, or Q levels that might 
exceed the N level. 


(2) Before initiating sampling at the 0, P, or Q level, require the product to 
"qualify" for the level by proving, at a 98% probability level, that the average level 
met by a sequence of lots was equal to or better than the level for which the product 
is to be qualified. 


(3) Specify that the lot is to be marked as meeting the level for which it is 
qualified only if statistical analysis proves that this lot and a specified number of 
lots that preceded it have averaged, at the 90% level of probability, at or above the 
level for which the product is qualified. 


Mathematical Computations 
Given: At the O level, P= -0195 and »° -00481 
Problem: Compute the reject line for a sequential sampling plan such that: 


(1) There is no more than a .05 probability of rejecting when lot 
reliability is PR, oO better. 


(2) There is at least a .90 probability of rejecting when lot 
reliability is P, or worse. 


Solution: The equation for the reject line is 
F=av+ dN 


In (1 + (1 -p) 
Where a= 


inp, - mp, + In (1 - p,) - In (1 - 


In (1 - p,) - In (1 - P,) 
And d= 


In py - In pot In (1 py) - In (1 Py) 
When the values P, = .05, P, = .90, Pp, = -00481, and p, = .0195 are 


substituted in the above equations, 
@ = 2.05 and b = .Oll 
and F = 2.0 + 


Equations for levels P and Q are derived in a similar manner by sub- 
stituting the values for Py and P) that apply to these two levels. 


The requirement describing the sequential sampling procedure for levels 0, P and 
Q, as it might appear in a specification, is illustrated in the following example: 


| 
| 
| 
| 
| 
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4.4.3 Sampling for Reliability Levels 0, P and Q. Samples for life 
testing at levels 0, P and Q shall be selected as for level N. 


4.4.4 Lot rejection. If the number of failures ds the pt 
ance number for the sample size, the inspection lot shall be rejected. 4 
Rejection shall be final and rejected lots shall be segregated to preclude 


mixing capacitors in any other lot. 


4.4.5 Reliability level identification. If a sample passes the life 
test and the capacitor is qualified at level M or N, the inspection lot 
shall be marked accordingly. If the inspection lot consists of capacitors 
qualified for levels 0, P or Q, the identification marking to be applied 
shall be determined in accordance with 4.4.6 and 4.4.7. 


4.4.6 Tabulation of life test data. For capacitors qualified at 
levels 0, P and Q, identification marking shall be determined on the 
basis of the results from a series of life tests. For this purpose, 
the results of life tests from both rejected and accepted inspection 
lots shall be tabulated, in the order in which the tests are performed, 
on a form similar to Figure 1. A separate tabulation shall be maintained 
for each style. 


4.4.7 Identification marking, levels 0, P and Q. When an inspec- 
tion lot is "accepted" by the life test sample, it shall be marked 
either at the level for which the capacitor is qualified or at the next 
lower level, depending on the process average for the last ten life test 
samples. After the data for the last sample tested is entered in the 
tabulation per 4.3.2, the totals for the last ten entries in the sample 
size and failures columns are summarized and the permissible number of 
failures computed by applying the appropriate equation in Table 7. If 
the actual number of failures exceeds the permissible number, the lot shall 
be marked at the level next lower than the one for which the capacitor 
is qualified. 


Table 7. Determination of Reliability Level 


Reliability Permissible Failures, F, for 
Level Total Samples Tested, N 
° F = 2.04 + .OO1N 
P F = 2.07 + .0026N 
Q F = 2.06 + .00066N 


Periodic Qualification Evaluation 

This element of the quality assurance provision is intended to ascertain whether 
the design of product from current production is capable of withstanding specified en- 
vironmental stresses. Periodic qualification evaluation consists of the tests in Group 
II, Environmental Tests. These tests can be performed periodically because they are in- 
tended to provide information regarding the capability of the design which, normally, 
is not subject to frequent or extensive changes. 


As an element of the multilevel reliability requirement, periodic qualification 
evaluation is designed to evaluate design improvement that increases the capability of 
@ part to withstand environmental stresses. Although the various component parts used 
in an equipment are subject to the same environmental stresses, existing specifications 
do not specify uniform requirements for these parts. The multilevel reliability re- 
quirement provides for such uniformity as indicated by the following examples of the 
new specification requirements: 


3.4 Environmental conditions 


3.4.1 Environmental conditions. Capacitors covered by this speci- 
fication shall operate satisfactorily after they are subjected to the 
following environmental conditions: 


\ 
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Table 3. Environmental Conditions 


Environmental Reliability Level 
Salt Atmo: re 
Hours 48 96 96 96 96 
temperature Cycling 
High °c 85 125 125 125 125 
Low % -55 -55 ~65 -65 -65 
Moisture Resistance 
Cycles 10 10 10 10 10 
Vibration 
Cycles/second 10-55 | 10-2000 10-2,000 10-2,000 | 10-2,000 
Acceleration (g) 15 15 15 15 
Shock 
Acceleration (g) 50 50 50 50 
Barometric Pressure 
Inches Hg. 3 1.31 0.315 0.043 0.043 
Altitude Ft. 50,000 | 70,000 100,000 150,000 150,000 


4.5 Periodic Qualification Evaluation 


4.5.1 se. The purpose of periodic qualification evaluation 
is twofold: (1) to detect any adverse effect on performance resulting 
from changes in material, design and construction detail; and (2) to 
verify superior performance capability indicated by life testing results. 


4.5.2 Qualification evaluation tests. Capacitors shall be sub- 
jected to the tests specified in Table 8 in the order show. 


Table 8. Periodic Qualification Evaluation Tests 


Tests Sample Acc. 
Size No. 


Subgroup 2 1 
Terminal strength 
Moisture resistance 6 1 
Shock 
Performance Tests 
Capacitance change 12 1 


4.5.3 Selection of sample. A sample of capacitors shall be taken 
from production each month for each style of capacitor in production 
during the month. The sample size for each test group shall be in ac- 
cordance with Table 8. 


4.5.4 Test method. The test method applicable to the level for 
which the product is qualified shall be as specified in Table 8. 


4.5.5 Test report. A report for each Periodic Qualification 
Evaluation test, certified by a responsible official of the supplier, 
shall be submitted to the qualifying agency together with the report 


| 
| 
| 
Environmental Tests 
Subgroup 1 
Barometric pressure 
Vibration 6 ‘ 
| Salt spray 
Temperature cycling 
| 
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required by 4.6.2. 


4.5.6 Initial test at higher level. When application is made for 
qualification at a higher reliability level, a test report shall be sub- 
mitted to the qualifying agency as evidence that the product has been 
subjected to and has passed the Periodic Qualification Evaluation test 
applicable to the higher level. 


Reliability Level Qualification 
Reliability level qualification is designed to serve two purposes. First, it pro- 


vides the means for determining before testing begins whether the part is designed for 
the reliability level it is expected to meet. Secondly, the controls built into the 
qualification requirement permit the use of very small samples for reliability conform- 
ance testing. The requirement, as it might appear in a specification, is illustrated 
by the following example: 


4.6 Reliability Level Qualification 


4.6.1 Purpose. The purpose of Reliability Level Qualification is 
to assure that the potential reliability of a design is equal to or higher 
than the reliability level it must meet to pass Reliability Conformance tests. 


4.6.2 Granting of qualification. A product shall be qualified for a 
higher reliability level when a summary of life test data obtained from 
Reliability Conformance tests indicates that the average reliability over 
a specified number of test hours is equal to or higher than the next level. 


4.6.3 Summary Report of Life Test Data. The supplier shall sun- 
marize life test data on a form similar to Fig. 2. A separate summary 
report shall be prepared for each style of capacitor produced by the 
supplier during the summary period. Summary reports shall be forwarded 
to the qualifying agency as of the last day of the calendar month. 


4.6.4 Amount of data in summary report. Table 9 shows the minimm 
emount of life test data required for a statistical determination that a 
product meets a reliability level. When an application is submitted for 
qualification at a higher level and insufficient life testing has been 
performed during the summary period to provide the amount of data called 
for by Table 9, data from preceding periods may be included in the sum- 
mary report. 


4.6.5 Application for qualification. Application for qualifica- 
tion at a higher level may be made whenever application of the appro- 
priate equation in Table 9 shows that the permissible number of failures 
is not exceeded. The application for qualification shall be accompanied 
by a test report of the Periodic Qualification Evaluation tests performed 
in accordance with 4.5.6. 


4.6.6 Revocation of Reliability Level Qualification. If the number 
of failures during any summary period exceeds the maximum number per- 
mitted by the applicable equation in Table 9, qualification shall revert 
to the next lower level. 


4.6.7 Extended life tests. At the option of the supplier, any part 
of the sample to be subjected to the life test may be designated for con- 
tinued testing in accordance with the Extended Life Test requirement, 4.7. 
Data from all extended life tests shall be included in the Summary Report 
of Life Test Data, and shall be used for the statistical analysis to de- 
termine whether the product should retain its qualification, lose it, or 
be qualified for a higher level. 
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Table 9. Control Limits for Qualification 
at Specified Reliability Levels 


Relia- Minimum No. Maximum No. of Fail- Maximum No. of Fail- 
bility of test hrs. | ures permitted dur- ures permitted dur- 
Level required ing T, total test ing T, total test 
for analysis | hours, to qualify hours, to retain 
qualification 

N 13,000 F= .0003T -.03/T F= .0003T 

0 62,000 F = .00008T -.c¥YT | F= .00008T +.cyT 

P 380,000 F = .00000T -.cOWT | F = +.00WT 

Q 840,000 F = .000005T -.oolVT | F = 


The control limits in Table 9 were computed in the following manner: 


Problem: oe an equation for computing the minimm number of failures, 
that can be expected during a total of T test hours if, at a 
34 level of probability, the average reliability of all lots 
tested is estimated to be equal to the 0 level. 


Given: N = Total Sample Tested 
t = Length of Test 
T = Total Test Hours 
F = Maximum Number of Failures in Test Time T 


Solution: Let } = the average fraction failures at Level 0. 
Then, at the 98% probability level, 5 = p, - 2VP} xa- P/N 
And, since p = F/N and N= T/t 


F = p,1/t - - 


Substituting P, = .0195 and t = 


F = .0195T/250 - 24/.0195 x .9805 x 1/250 
= .000078T - -0175-/T 
Note: This equation appears in the revised version of MIL-C-25 Table 9. 
as F= .00008T - .c2 


Problem: Compute the minimum number of test hours that must be accumulated 
in order to determine whether a product qualifies for the 0 Level 
of reliability. 


Solution: T is minimum when F = 0. 
Substituting 0 for F in the equation, F = .00008T - .02-/T, 
O = .00008T - .02-/T 
And T = 62,500 hours 
Note: This value for T is shown in Table 9 of the revised version of 
MIL-C-25 as 62,000 hours. 


Problem: Derive an equation for computing the maximum number of failures, 
F, that can be expected during a total of T test hours if, at a 2% level 
of probability, the average reliability of all lots tested is estimated 
to be equal to the 0 Level. 


Solution: Let p = the average fraction failures at Level 0. 
Then at the 2% probability level, = py + 2+/P)(1 - 
And, since = F/N and N = T/t 
Substituting = .0195 and = 250 
-00008 T + .02./T 


Note: This equation appears in the revised version of MIL-C-25 as the 
qualification revocation equation. 
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CONCLUSIONS 

The multilevel reliability requirement offers a practical approach to the problem 
of incorporating reliability assurance provisions into part specifications. A specifi- 
cation embodying such a requirement does not require periodic updating as the state of 
the art advances inasmuch as requirements pertaining to higher levels of performance are 
included in the specification and can be applied as soon as these higher levels are at- 
tained. Such flexibility is desirable both to the manufacturer and to the user of a 
part. The former can obtain recognition for significant improvement in the performance 
of his product, while the latter can determine without additional testing whether parts 
are being manufactured that have the performance characteristics he desires. 
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EFFECTIVE QC INCREASES POLARIS RELIABILITY 


Francis A. Gall, Manager, Quality Assurence 
General Electric Company 
Light Military Electronics Department 
Utica, New York 


Much publicity has been given to the Polaris program in many fields. Without 
doubt much of this is deserved since the Polaris not only represents the first opera- 
tional inertially guided missile, but, what may be more important, the accumulation 
into one deterrent weapon of the latest technological advancements in many fields in- 
cluding navigation, sonar, communication, ship building, propulsion, etc. Yet de- 
spite the size of the task, it was accomplished successfully on e crash basis in 
record time. 


In order to cope with the speed of the program, quality control methods had to 
be devised that were simple and quick on the feedback. This paper will describe two 
methods, one used in inspection and another used in test, that proved highly success- 
ful in controlling the workmanship and reliability of this fast moving project. They 
are described not because they are brilliant new approaches nor because they are pan- 
aceas, but rather because of their simplicity and adaptability to any program. 


The first of these two, a method for controlling workmanship during assembly 
came about through the need for identification of operators who required more train- 
ing. The program represented a new standard of workmanship and required more careful 
work because of the compactness of the components and also the use of semi-conductors. 
To make matters more difficult there were in the guidance and fire control systems 
numerous types of units and low production quantities. This resulted in a very 
heterogeneous group of assembly operators. The problem was to obtain a quality report 
that would sum up this type of operation in terms of quality or workmanship. 


The common elements in all of the work performed were soldering, crimping, dirty 
joints, missing parts, etc. (See Figure 1 for typical code list.) Therefore, we set 
about to list weekly the number of defects reported by the inspectors for each opera- 
tor. In addition, like defects were totalled and a weekly report was issued of the 
most prevalent defects in the assembly area by foreman. Figure 2 shows a typical 
weekly report indicating operator check numbers and defect types. It will be noted 
that not every operator delivered work to inspection each week and that conversely 
if they did and no defects were found this was recorded by a zero. In this manner 
each week a ratio of the total number of defects in the area to the number of opera- 
tors turning in completed work could give a figure for an average number of defects 
per operator. A plot of this ratio is shown in Figure 3 showing the dramatic results 
achieved during the first six months of its inception. Although this measure may seem 
arbitrary, through usage and its own simplicity it has been a very good yardstick. 


In the implementation of this plan, however, the work of the different operators 
cannot be absolutely compared since one may work on a harness consisting of several 
hundred connections while another may work on a chassis consisting of many fewer 
connections to delicate semi-conductor elements packed together very closely. Also, 
there may be several new operators in a particular area making it nlikely that the 
foreman will appreciate being compared to other foremen. One ti.ng, however, that can 
be insisted upon and on which much of the success of the plan depends is that each 
manufacturing foreman use the reports to improve the performance of those operators 
that have the greatest number of defects and if necessary transfer them to a more 
suitable job if they cannot attain an acceptable level of workmanship. At the begin- 
ning of the program, especially, three characteristics were necessary in an operator 
to be successful: accuracy, speed and versatility. The first two are obvious, the 
third may require a word of explanation. Many changes were introduced in rapid suc- 
cession and very few operators worked on the assembly of only one type of unit, some 
of which took many days to assemble. It was, therefore, very important that operators 
with the proper temperament be selected for the project. 


By the end of the first quarter of 1960, it was obvious that a plateau had been 
reached and something had to be done to further reduce the number of defects per opera- 
tor. It was decided to have periodic meetings between assembly and quality control 
foremen to discuss ways of reducing defects further. Several worthwhile corrections 
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in the process resulted. 


For example, damaged parts in assembly were excessive. Through discussion at 
the meetings, i+ was learned that the major cause of this defect was the fragility 
of traasistor leads. A handling program fcr these parts was instituted throughout 
the whole planv vith gratifying resiits. However, it was not until someone suggested 
building up the base of the transistox with some type of epoxy to move the flexing 
peint of the leads beyond what ~as b<ileved to be a length of the leads partially 
oxidized in the vendor's ope siies tat the problem was solved. Addition of this 
step ir once wmerats rtually eliminated any lead breakage problems in our 
plant 


in another case, differences i-tween the operators and the inspectors as to the 
mowit of solder on a joint were tl. * to be the reason why so many connections were 
called because of "insufficient solde:'. To our surprise the answer turned out to be 
the inadequate thickness of the gold plate on the connector terminals and the result- 
ant corrosion. This prevented solder flow and adherence of the solder. As a result, 
negotiations with the vendor cleared up the problem and today it is virtually non- 
existent. 


Yet, despite such progress the defects per operator were not coming down. 
Apparently we were not getting to the operators. Near the end of the year an inten- 
sive program was started in this direction. This time all the operators and the 
inspectors were given directions on the correct methods of performing those three or 
four operations that in past weeks accounted for the majority of the defects. The 
weekly bar chart shown in Figure 4 illustrates how the topics were selected. The 
discussion between operators and inspectors at these meetings proved stimulating and 
worthwhile. Immediate improvement in workmanship was noted the following week by a 
downward trend in the curve and in January of this year the lowest number of defects 
per operator as of that date was reached. We had cut defects to one sixth in a year 
and a half! 


To further impress the operators with the need for good workmanship, posters are 

laced in the work area relating to the major points covered at the last meeting. In 
this manner it becomes more than a one hour discussion and impresses the ideas daily 
on the minds of each operator. A colored chart is also posted in each foreman's area 
listing ail his operators and relating their performance each week to a color codd. 

A quick glance at the chart will reveal through the color pattern whether the fore- 
man's area is within the goals set, whether he is beginning to be in trouble or is 
already completely out of control. It might be added that at any time the foreman 
can request daily feedback for their areas and the tabulations can be made available 
by the next day. 


The second method improved reliability in a more intangible way through a con- 
siderable reduction in trial and error troubleshooting during test. This method, 
called a test plot, shows the hourly progress during a particular test on a single 
unit. 


Figure 5 shows a typical test plot and demonstrates the upward trend of good time 
during which the test is progressing according to the test procedures. The downward 
trend of the curve represents bad time during which additional unexpected problems 
are being located and resolved. The code numbers under the bad time identify the 
probable nature of the problem. The horizontal lines indicate delays due to rework, 
changes, shortages, etc., not directly under the test area jurisdiction. Since this 
chart is kept hourly by the tester it can be seen at a glance without disturbing any- 
one what the progress of the particular unit happens to be through the test. Reading 
the code underneath the chart indicates the nature of the problem and identifies the 
type of support assistance that may te necessary if too long a delay is observed. If 
the downward trend extends for more than a few hours the foreman can immediately re- 
view the situation and call in additional help before too many hours are wasted. This 
procedure has expedited many tests since often the test equipment or design engineers 
understood the limitations of the circuitry better than the testman and were also 
aware of design changes that may not yet have been integrated in the test equipment or 
the procedures due to the rapidly moving program. In a short test of a few hours dur- 
ation this technique of test plot would not prove very useful, however, it should be 
remembered that these tests of which we are speaking were in some case a few hundred 
hours in duration and, therefore, some indication of the progress was necessary to 
accumulate and summarize the total test picture of the unit. 
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The code assigned to the bad time has nine separate categories. Briefly, they 
are workmanship errors, engineering design problems, fabrication problems, test equip- 
ment problems, selling problems, engineering changes, component failures, test pro- 
cedure problems and test personnel problems. Through this identification of the prob- 
lems corrective action can be initiated very quickly at any particular time for a 
given unit in test and through casual perusal of the test plot recurrent failures of 
a particular category can be identified for their effect on the program. For example, 
the effectiveness of the complex universal test equipment used to test the fire con- 
trol units could be evaluated from the amount of bad time necessary to correct test 
equipment problems. 


The usefulness of these test plots was not limited to the very short, almost 
instantaneous, feedback time on the factory floor. The test plots are equally useful 
in predicting schedules by reviewing trends from the previous tests and also for quot- 
ing. In both cases the decisions can be substantiated by reference to the test plots 
and improvement factors can be included based on projections of the learning curve. 
The learning curve for the first ten systems is shown in Figure 6. It has a very 
rapid decrease in test time to the end of the third fire control system. We like to 
attribute some of this increase in productivity to the test plot method and its quick 
reaction time. 


It might be interesting to note that this method is of very little use in the 
test area today since all but three units require less than 10 hours test time. The 
remaining three total about 150 hours to complete test. Even so, the problems are 
mostly of a component nature and require very little support experience. 


The simplicity and the effectiveness of these two techniques lie in the fact that 
all of the records are made on the spot in test and inspection at the time when the 
information is most familiar to the people who are actually doing the work; no elab- 
orate mathematics or analyses are necessary. The cycle time for the use of the data 
is no more than one day in the case of the workmanship reports and a matter of instan- 
taneous usage in the case of the test plots. Both reports not only furnish immediate 
information that can be applied to nearly every program but they also are useful at 
higher managerial levels to observe overall aspects of operations. 
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TYPICAL DEFECT CODE LISTING 


005 INSULATION DAMAGED - MECH 153 DEFECTIVE LACING 
132 EXCESS SOLDER 154 DAMAGED FINISH 
133 INSUFFICIENT SOLDER 157 STALKING 
137 BURNED WIRE 163 INSUFFICIENT SOLDER FLOW 
138 POOR CRIMPING 221 DEFECTIVE WELD OR SOLDER 
144 MISSING PART 230 DIRTY 
147 INSULATION BACK TOO FAR 340 INSTALLED IMPROPERLY - WIRING 
149 DIRTY SOLDER JOINTS 731 AUTOMATIC CONTINUITY CHECKER 
FOREIGN MATERIALS 750 MISSING COMP HARDWARE OR RUN 
152 BROKEN WIRES 848 WRONG PART - ELECT VALUE 
Figure 1 
TYPICAL OPERATOR DEFECT SUMMARY 
WEEK NOS. OPER. NO. K 

1137 5 17 4] 0/0 
2942 10 | 23/6 6/14 | 4571 ° 
4195 5 2 |19| 5047 le | | 
4339 2 | 10/12 sors | o| 
4380 3/2 5232 
4431 9 | 14/12 29|10 16 | | se75 || 3 
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AVERAGE WEEKLY NUMBER OF 
DEFECTS PER OPERATOR 
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TYPICAL TEST PLOT 


24 HOUR DAY: 
9-28 9-29 9-30 10-1 10-2 
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Figure 5 
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QUALITY CONTROL AND RELIABILITY ORGANIZATION 
A PROFESSIONAL APPROACH * 


Sigmund P. Zobel, Manager, Quality Control 
The Carborundum Co. 
Electro Minerals Division 
Niagara Falls, N.Y. 


ABSTRACT 
This paper attempts to provide an insight into the problem of se- 
lecting an organizational structure to provide professional level qual- 
ity control and reliability activities for the industrial unit. Con- 
sideration is given to organization as a function of corporate structure, 
size of manufacturing unit, nature of products, and nature of markets. 


INTRODUCTION 

Historically, the manufacturing establishment had a going inspec- 
tion department as its principal quality control function. With this lLim- 
ited scope of interest there were few problems of organization, if any. 
A necessary service for the plant manager or factory superintendent was 
being performed. However, with the advent of the application of statis- 
tical principles, first in support of the inspection function, then to 
facilitate in-process control, and now to provide valid reliability fac- 
tors, the matter becomes increasingly complex. Dr. Romig summed it nice- 
ly in his 1953 Shewhart Medalist address, when he said: 

"The principles of statistical quality control embody the 
applications of the scientific method to all phases of our indus- 
trial life including management, research and development, pur- 
chasing and vendor relations, plant evaluations, production en- 
gineering, inspection, test, accounting and industrial relations. 
It takes certain of the operations research techniques used by 
the military and develops scientific teams that are copgjdered 
by many managements as their most valuable adjuncts." 

If these words may be regarded as a charge, it is easy to see why the 
task of organizing to fulfill the implied mission is indeed a formida- 
ble one. And, it may be added, is seldom completely consummated. 


It is the purpose of this paper to give the reader an increased 
awareness of the need for a sound professional approach to the matter 
of quality control and reliability organization. In the sequel, typical 
management dilemna arising in this connection will be discussed. And, 
the importance of human relations will be stressed. It is well known to 
all quality control practitioners that the problems generated by the 
human relations factors not only affect the degree of success, but may 
also shape the organizational structure of the quality control and reli- 
ability functions. Throughout the discussion, the writer subscribes to 
the opinion that quality control and reliability are inseparable func- 
tions, and both should be under the ultimate control of one group. 


ATTACKING THE PROBLEM 
Now, let us consider ways in which one may attack the problem of 
creating an organization. 


From the beginning, it is necessary to have a clear definition from 
top management of its policies, objectives, and degree of financial 
support that will be forthcoming. The odds are against a successful 
build-up of a professionally competent quality control and reliability 
organization without such a foundation. If it is not readily obvious 
why this is the case, a quotation from Feigenbaum, and subsequent con- 
sideration of its implications, may facilitate an appreciation of this 
need. In discussing the nature of an effective quality control organ- 
ization, Dr. Feigenbaum says, 
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"It is a device whereby management delegates authority and 
responsibility for product quality, thus relieving itself of unnec- 
essary detail and permitting the benefits of specialization, yet 
retaining for itself the means for assuring that quality results 
will be = in terms of top management's standards and 

2 


policies." 


We may note that the quality control organization is the seat of 
management-delegated authority and responsibility for product quality. 
For this to be so in fact, and to enable the quality control and reli- 
ability group to discharge effectively its responsibilities, the organ- 
izational structure must give to part of the group at least minimal line 
authority. Responsibility and authority are the veins and the arteries 
of any function within the corporate body. Just as the heart cannot per- 
form its function without veins and arteries, neither can the quality 
control group. Authority commensurate with the responsibility must be en- 
joyed, or the quality and reliability responsibilities cannot be ful- 
filled successfully. At this stage in our discussion, it may be well to 
take cognizance of some of the more important responsibilities, since 
they are a guide to the organizational structure, and to the kind of tal- 
ents needed in the group. Once again, the writer will draw upon Feigen- 
baum. He lists the following as among the more important responsibilities. 

"lL. To aid management in the preparation, adoption, and 
continued evaluation of a factory-wide program of quality con- 
trol, expressed in terms of written company policy. 

2. To advise line, staff, and functional groups in man- 
ufacturing and Engineering as far as their quality control 
activities are concerned. 

3. To develop and to maintain for top management an effec- 
tive product-quality appraisal system. 

4. To stimulate quality research. 

5. To maintain or to stimulate quality-control education. 

6. To aid management in stimulating a quality-minded 
attitude throughout the plant. 

7. To assist - and, in some cases, to act for- manage- 
ment in the actual details of administering and coordinating 
the factory-wide quality-control program. 

8. To establish and to perform certain quality-control 
services. 

9. To advise management in the product-quality aspects 
of the establishment of new plants and manufacturing areas. 


10. To aid Marketing and Sales in its merchandising and 
+ aaa activities and in product advertising cam- 
paigns. 

11. To develop a competent and efficient group of men 
and women to carry on the quality-control staff functions." (3) 


i To these should be added an additional responsibilty 
or two: 

12. To put "statistical" in front of "quality-control" 
whenever appropriate. 

13. To generate a plant-wide reputation for fair, un- 
biased solutions to its problems. 

14. To keep up to date in the pertinent applications of 
statistics and mathematics. 


These fourteen do not comprise the full set of responsibilities a 
quality control group may be given. They do, however, serve the purpose 
of providing a framework within which to operate while considering the 
type of organization needed, and the type of personnel required. 


As Feigenbaum also points out, all of these responsib i 
functions of a small number of general issues: ee ee 


1) How is quality control related to the ins ection f 
tion? 
2) Should the quality control staff have an Pair an -q 
responsibilities? 
3) How is the staff to be organized? 
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4) What is the status of the staff and its manager or director? 


Resolution of these issues, and assignment of the responsibilities 
will be discussed in the next section, where various forms of organiza- 
tion are considered. As various possible structures are being explored, 
the reader should constantly remind himself that quality contains U, the 
precious ingredient without which there can be no quality, no reliability, 
no group effort. But the U must be both singular and plural, or else it 
vanishes literally and figuratively. Accordingly, a useful, workable 
organization must recognize the community of interest, and provide, formal 
or informal means for coordination all pertinent interests. 


Now, what interests may be recognized as relevant? A clue may be 
obtained to a first appreximation to the answer by a brief examination 
of the Quality Control Wheel, Illustration 1. 


MANUFACTURING 


THE QUALITY CONTROL WHEEL 
Illustration 1 


(4) 
While a more complete discussion of this concept may be found elsewhere, 
a cursory treatment is germaine to the present purpose. It can be seen 
quite readily that the quality control function is a focal point. Through 
the formal and informal organizational lines of communication, there must 
be rapport between quality control and sales, between quality control and 
manufacturing, between quality control and design, and among sales, de- 
sign, and manufacturing. 


The quality control function at the hub of the Wheel should consist 
5 


of three main elements, after Juran, as discussed by Scott. These ele- 
ments are Inspection, Statistical Quality Control, and Quality Assurance. 


The first is concerned mainly with the manufacturing part of the 
Wheel, since it deals with receiving inspection, process inspection, fin- 
ished goods inspection, gage maintenance, test equipment maintenance, 
salvage, day-to-day trouble shooting, etc. These activities may be termed 
the acceptance function. 


Statistical quality control, the prevention function, includes sta- 
tistical methods, training in quality control, analysis of data, process 
capability studies, economic studies, design of sampling plans, and 
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design of experiments. It deals mainly with the manufacturing and the 
design functions. 


The third, quality assurance, covers customer complaints, quality 
audit, check inspection, market quality determination, accuracy of in- 
spectors and inspection methods, executive reports on quality, quality 
certification, reliability activities, and responsibility for valid spec- 
ifications which have been subscribed to by all parties concerned. This 
function, therefore, is largely tied in with sales, but still must work 
with manufacturing and design. 


At this point, the reader should be ready to consider a few of the 
most likely organizations. 


SOME ALTERNATE ORGANIZATIONS 
In this section, several different organizational structures will be 
considered. Their merits and disadvantages will be discussed to a limited 
extent, with the hope that the interested reader may be provided some 
guidance in coping with his own organizational needs. The keynote > 


discussion to follow can be taken from a few remarks of Charles A. Bicking 


"The organization should follow the sound principle that who- 
ever is responsible for achieving proper quality in manufacture 
should not also be responsible for what constitutes proper 
quality against which the achieved level will be judged. 


The exact organizational chart is not important and may vary 
from company to company. There must be a clear separation, 
however, of the function of manufacturing quality into the 
product and the function of prescribing the quality standards 


and assessing the degree of success achieved in meeting the 
standards." 


A, A Completely Decentralized Or Single Unit Establishment 


We may begin with the simplest case, and build upon this, until we 
have developed as complex a case as may be suitable for the present 
paper. In this, and the following, section(s), the thinking and writings 
of several students of the subject, in addition to the writer, will be 
synthesised. Rather than attempt to give specific reference each time, 


the sources drawn upon in varying degrees will be cited in the references 
below. 


Obviously, where there is only one physical plant, be it a complete- 
ly decentralized plant of a multi-plant company, or a single unit estab- 
lishment, the same considerations obtain. Illustration 2 indicates one 
possible organization for this case. 


General Manager 


| | 
isles Design Manufacturing Finance 


and Reliabilit 
rocedures 
Inspection 
Quality Assurance 
Reliability 


Line Supetintendents 


Case A 
Illustration 2 


Only the area of immediate interest, quality control and reliability, is 
elaborated upon. In this, and in the illustrations to follow, listings 
which appear to be functional rather than titular do not necessarily im- 
ply separate personnel. Indeed, several activities may be performed by 
one individual. 


| 
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Case A is an organization of the sort that may have evolved, or 
“just growed". The nature of the chief ins'pector's job probably changed 
as a function of competitive pressures and/or a growing awareness of the 
changing character of the quality control function. He is now the quality 
control supervisor or manager, reporting to the manager of manufacturing 
or plant manager. The dotted line represents a communication line (as-it 
also will in subsequent illustrations) which must exist, or the preventie 
function will not be operative. 


If, now, we recall the keynote of this section, it is readily rec- 
ognized that this form of organization violates the principle set forth, 
with regard to the separation of functions of achieving the specific 
quality, and measuring the degree of achievement. Applied statisticians 
are quite familiar with the term, bias. We use random sampling, labora- 
tory controls, blind samples, etc. to eliminate or minimize it. Would it 
not, then, be anamolous to countenance an organization conducive to bias, 
while we disclaim it in our estimation of process parameters or in analyt- 
ical results? 


An organization such as that presently under discussion has an in- 
herent bias. If a group sets standards by which to appraise its own 
efforts, it would be quite natural for the standards to be applied and 
interpreted in such a way as to put the manufacturing quality control in 
a somewhat more favorable light than may actually be the case. This may 
be called, to coin an expression, proprietary bias, or the bias due to 
defending one's own work. There is no real quality conscience in this 
set-up. And, the void can only be filled by a clear separation of func- 
tions, as recommended above. To accomplish this, the organization depic- 
ted as Case A need only be modified somewhat to result in Case B 1, as 
in Illustration 3. 


General Manager 


T ] | 
Sales Design Manufacturing Finance 
/ 
/ 
1 | Supt. ty 
Case Bl 


Illustration 3 


There is now a conscience. The manufacturing quality control group 
has the responsibility (and, it is to be hoped, sufficient authority) for 
in-process and final product quality control, including inspection, and 
for recommending materials and processing changes in the interests of 
maintaining or improving quality. On the other hand, the staff group has 
the responsibility for setting the standards against which the product is 
evaluated, and serving as the management agent. Their responsibilities 
include, as a partial —— only, specifications, quality control pro- 
cedures and standards, auditing the manufacturing quality control results, 
keeping management informed, making reliability studies, and acting as 
liaison between sales, finance, and design on the one hand, and manufac- 
turing on the other.*It should be obvious to the reader that both groups 
are needed to fulfill the mission as indicated above, in terms of the 14 
responsibilities which were enumerated earlier. Since the special skills 
necessary to implement the manufacturing quality-control function are 
seldom present in the same personnel who possess the special skills req- 
uisite for the staff quality control function, the groups can rarely be 
combined with success. Incidentally, this is another reason why the or- 
ganization in Case A can seldom be truly effective to a high degree in 
terms of quality control and reliability. 


| 
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A variant of the same basic organization, and one which in many 
situations would be more effective, is presented in Case B 2, Illustratim 
4, without further discussion. 


General Manager 
1 


Sales Manufatturi Technical Finahce 
Man ng finance 
| Res, and Eng. | 

Case B 2 


Illustration 4 


B, A Multi-divisional Establishment, With Centralized Policy Making 


In a multi-divisional or multi-plant situation, a somewhat different 
organization is ordinarily required. However, there may be such complete 
decentralization and autonomy that each plant or division may establish 
its own policies independently of over-riding corporate policies. In 
this event, Case B 1 or B 2 should adequately provide for a meaningful 
quality control and reliability organization. But, more often than not, 
there will be corporate policies which provide the context and restric- 
tions for divisional activity. When this is the case, the optimum organ- 
ization for quality control and reliability is one which, when viewed 
superficially, appears quite complex. However, the very characteristics 
which create the impression of complexity can give it real strength. 


At the corporate staff level there should be a group who, by their 
experience, training, and capabilities can be truly regarded as profes- 
sional in the fields of quality control, reliability, and applied indus- 
trial statistics. This group, acting as the agent of the top management 
of the company, establishes policies, provides consultation to the plant 
or divisional staff quality control and reliability groups, and audits 
the pertinent activity of each plant or division. There must be a direct 
line of communication between this corporate staff group and each local 
staff quality control and reliability group. And, this top level staff 
eo must act as the quality conscience for each of the plants or div- 

sions, and the entire company as well. 


Before leaving this section, the author wishes to point out that 
only two or three situations have been discussed, and to a rather limited 
extent at that. But, a full treatment of the subject cannot be rendered 
in anything as short as a paper of this nature and for this purpose. 


CONCLUS ION | 
Rather than attempt a summary of the foregoing discussion, since 
the subject was so briefly touched upon, we will mention a few of the 
corporate ramifications that have a bearing on the present subject. Any 
architect of a quality control and reliability organization must take 
these into account if he is to build an enduring edifice. | 


Some consideration has already been given to the corporate structure, 
in terms of decentralized versus centralized control. Size of the manufao 
turing unit, in personnel and sales volume, must also be considered. A 
small unit, under the close personal control d the general manager, can 
not tolerate the comparatively more complex organization required to fill 
the need in a unit where the demands on the general manager preclude his 
close personal control. 
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Further, the nature of the products have a bearing on the most 
appropriate form of organization. For example, in the electronics, arma- 
ments, and missile industries, the entire quality control organization 
must be pointed towards reliability. On the other hand, a producer of 
bulk materials must have his quality control organization oriented to- 
wards quality assurance. 


Another factor which may influence the quality control and reli- 
ability organization is the importance of government contracts in the 
market of the company. Also included in the nature of the market is the 
extent to which the praiucts are end use items, or are by way of being 
producers' goods, which are raw materials to the customer. Consequently 
varying proportions of terminal product and non-terminal product, from 
one company to another, also have considerable influence on the nature 
and place of the group charged with the quality responsibility. 


No matter what the organizational structure may be, or at what 
levels the quality control responsibilities and authorities are placed, 
certain cautions are pertinent. 


Such a group must be flexible, somewhat informal, and in possession 
of appropriate talents and skills, Not the least of these is an ability 
to satisfy simultaneously the conflicting demands which frequently arise, 
such as between sales and manufacturing, or design and manufacturing. 
Further, the precepts of sound organization, such as the awareness of the 
limits of the span of control that any one executive can exercise effect- 


ively, and cognizance of the need for a proper balance between line and 
staff functions, must be followed, 


The ultimate test of a sound quality control and reliability organi- 
zation is, does it make it possible to accomplish its mission? Therefore 
it should be designed with the mission in mind. A professional approach 
to creating the organization must start with a clear understanding of 
management's concept of the mission, and must consider the inter-relation 
ships that will be present between quality control and reliability acti- 
vities on the one hand, and all other functions of the plant or division, 
on the other. 
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AN ADAPTATION OF THE 
MIL-STD-105B PLANS TO RELIABILITY 
AND LIFE TESTING APPLICATIONS* 


Henry P. Goode and John H. K. Kao 
Department of Industrial and Engineering Administration 
Sibley School of Mechanical Engineering 
Cornell University, Ithaca, New York 


SUMMARY 


This paper presents a procedure, together with tables of necessary conversion 
ratios, for applying the MIL-STD-105B sampling-inspection plans to reliability and 
life-testing applications. The method assumes the Weibull distribution (including 
the exponential distribution) as a statistical model for item lifelength. Inspec- 
tion of sample items is by attributes with life testing truncated at the end of 
some specified time, t. Lot quality is evaluated in terms of mean item life, np. 
Both t and p are measured from some reference time, 7. 


INTRODUCTION 


This paper represents an extension of a paper and set of sampling plans for 
reliability presented by the authors at the Seventh National Symposium on Reliability 
and Quality Contre‘), Both these papers are generalizations of papers by Sobel 
end Tischendort*) ena ty Rpstein'>) thet appeared respectively in the Proceedings 
of the Fifth and the Sixth National Symposium and by Altman 4 that ared in 
Industrial Quality Control. Plans have also been prepared by Myers 3 and related 


work done by Gupte and Groll 6 - However, in this paper and the one to which it is 
related the Weibull form for the underlying life density is assumed whereas in the 
other papers cited the exponential model (or the gamma variable which is the sum of 
exponential variables) is assumed. 


Numerous studies of lifelength for a variety of materials and components have 
shown the Weibull model to be an excellent one to use as an approximation to the 
true distribution of life. The exponential model, which has been commonly used up 
to this time, requires the assumption of a constant failure rate regardless of the 
life of the items while the assumption of a gamma distribution as the failure model 
has the characteristic that as items increase in age, the failure rate tends to a 
fixed value. These assumptions may in many cases be far from realistic, particularly 
when failure is of the wear-out type. It has been demonstrated that sampling 
inspection decisions based on the assumption of the exponential are highly distorted 
if the true underlying distribution of life-length is of the Weibull form. 


The objective of this study is to present a working procedure, together with 


tables of necessary conversion ratios, for applying the MIL-STD-105B sampling 
inspection plans to reliability and life-testing applications for which the Weibull 
model is appropriate. A table is also included for the special Weibull case (6 = 1) 
for which the Weibull is the same as the exponential. Thus both possibilities for 
a statistical model for life-length are covered. 


THE SAMPLING INSPECTION PROCEDURE 


The following acceptance-sampling procedure has been adopted for the adapta- 
tion considered here of the MIL-STD-105B plans for life testing: 


(1) A random sample of n items is selected from the lot. 


*This research was supported by the Office of Naval Research. Reproduction in 
whole or in part is permitted for any purpose of the United States Government. 
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(2) These sample items are tested for life over some preassigned test 
period t units in length. 


(3) The number of test items that fail prior to time t is observed. 


(4) If the number of items failing is equal to or less than some specified 
acceptance number, c, the lot is accepted; if the number that fails 
exceeds the acceptance number, the lot is rejected. 


The sample sizes and acceptance numbers used in the above procedure will be those 
specified by the MIL-STD-105B tables. The procedure as outlined above is for single 
sampling; through simple and appropriate modification, the 105B double-sampling and 
multiple-sampling plans may be likewise employed. It may be noted that the acceptance 
procedure is the same as that specified for the MIL-STD-105B plans with the single 
exception of the use of a test truncation time, t, (and the application, of course, to 
life-testing data). 


The probability of acceptance for a lot under the acceptance procedure outlined 
depends solely upon the probability, p', that an item will fail before the end of the 
test period, t. One may note from a study of the mathematics as outlined in the 
Appendix that if the test truncation time, t, is preassigned and if the value for 6, 
the shape parameter, is know the probability, p', of failure is a fumction only of 
mean item life, ». This fact makes it possible to use this attribute acceptance pro- 
cedure to evaluate lots in terms of mean item life; the operating characteristics for 
any specified plan (in terms of c and n) depend only on ¢ and np. 


In order to allow for any desired test-time truncation value, t, and thus provide 
Plans for general use, the tables included with this paper have been prepared in terms 
of the dimensionless ratio, t/u. Actually, to give more conveniently usable figures 
and to work in terms adopted for the 105B plans, the ratio is given in terms of percent 
(t/u) x 100 is used. Each of the 105B plans is cataloged and described in terms of 
the (t/p) x 100 ratio. These ratio values are used in the same way as the percent 
defective values are used in the selection and application of the 105B plans for ordi- 
nary attribute inspection. When applying the plans to a specific life-testing applica- 
tion, use of the ratio to convert from test time to lot mean (or vice versa) will be 
found quite easy. Examples of application will be given near the end of the paper. 


It will be noted that it is necessary to assume a value for B, the Weibull shape 
parameter. For many applications this value may be known. Its magnitude may have 
been determined for the product in question from past life-length research data, from 
the results of past inspection date, or from some other source. If the value for B is 
not known, procedures are available for estimating this parameter (and the 7 location 


parameter if this, too, is necessary) 9) 


THE TABLES OF RATIOS FOR THE 105B PLANS 


Tables of ratios for adapting the 105B plans to reliability and life-testing 
application have been prepared for each of seven values for 6, the Weibull shape para- 
meter. These seven values range from 1/2 to 3-1/3, covering the span commonly 
encountered with industrial products. A table for B = 1, which is the exponential 
case, is also included. These tables will be found at the end of this paper as Tables 
1-A through 1-G. 


Each table lists for each 105B Acceptable Quality Level (AGL) value the correspond- 
ing (t/n) x 100 ratio. These matched ratios will be found in the column headings under 
each of the respective 105B Acceptable Quality Level values (which are in terms of 
100 p' %, the acceptable percent defective). Each ratio value gives for all 105B plens 
of the corresponding AQL, a measure of lot quality for which the producer's risk or 
probability of rejection will be low. This risk of rejection will be the same as thet 
encountered in the normal use of the 105B plans for attribute inspection. I+ will be 
recalled that this risk is not a constant value of, say .05, as in most previous 
tables of acceptance inspection plans, but ranges from as low as 0.01 to as high as 
0.20. The risk varies with the size of the sample, which in turn varies with the size 
of the lot. For large lot sizes (and thus large sample sizes) the risk is relatively 
small; for small lot sizes it is relatively large. The specific risk value for any 
plan of interest may be obtained from the corresponding operating characteristic curve 
which will be found included with the 105B tables. 
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The interpretation of these matched Acceptable Quality Level ratios for life- 
testing and reliability use may be demonstrated by means of a specific case. Assume, 
for example, that 6 = 1-2/3 and that a 105B plan with an AQ@ of 4.0% is to be used. 
From the table of ratios for 6B = 1-2/3, which is Table 1-E of those included at the 
end of this report, it will be found that the corresponding t/p ratio value at the 
AQ is 16.42. Thus lots for which (t/y) x 100 = 16.42 are “acceptable” and the pro- 
bability of acceptance will be high (the probability of rejection low). If the test 
period, t, is, say, 1000 hours, (100/u) x 100 = 16.42 or » = 6,090 hours; the mean 
life for the items in the lot must be 6,090 hours for it to meet the 105B acceptable 
quality level standards. 


In the body of each table of conversion ratios will be found, for each 105B plan, 
ratios for which the probability of acceptance is .10. These correspond to lot 
tolerance percent defective (LTPD) figures which furnish a useful measure of con- 
sumer's protection. They represent unsatisfactory lot quality values and ones for 
which the probability of acceptance is low. Unlike the risks associated with the AGL 
which vary with sample size, the risk at the LTPD quality used for the tables 
presented here is at .10 for all plans. 


For the example cited above in which the A@ is 4.0%, suppose the Sample Size 
Code Letter is L. Reference to the table of conversion ratios for 8 = 1-2/3 shows 
the LTPD ratio to be 31. With the test period, t, designated as 1000 hours, the 
value for uw, the lot mean life may be readily determined. Substitution gives 
(1000/m) x 100 = 31 or p = 3,220 hours. Thus lots whose mean life is 3220 hours or 
less have a probability of at most .10 of acceptance. 


With the use cf these complete tables of conversion ratios (tables 1-A through 
1-G), suitable 105B plans may be selected in terms of either an AQ or the LIPD. If, 
instead, some 105B plan has been specified, its operating characteristics can be 
evaluated. Examples of such use will be outlined in the material that follows. 


As a supplement to these tables, Table 2 has been prepared. This table gives the 
(t/p) x 100 ratio at the Acceptable Quality Level for an additional number of values 
for B, the Weibull shape parameter. Ratios at the A@ for the 6 values used in 
Table 1 are also included. As the Acceptable Quality Level supplies the operating 
characteristic of most interest in the application of 105B plans, the ratio values in 
this table may be all that are necessary for many applications. 


Example (1) 


For a simple example of application, consider a receiving inspection cease for 
which incoming lots of a product are to be tested for lifelength by sampling. From 
past experience with the product it has been determined that the life distribution 
can be expected to follow the Weibull form with a value for 6, the shape parameter, 
of approximately 1-1/3. The value for 7, the location parameter, is expected to be 0. 
The MIL-STD-105B plans are to be employed. A test period for the sample items of 
200 hours and an Acceptable Quality Level in terms of percent defective (as used in 
the Standard) of 1.5% have been more or less arbitrarily selected for use. The size 
of incoming lots is 5,000 items. Inspection Level II, the one for normal use, will 
seemingly be appropriate. Single sampling is to be employed. The acceptance pro- 
cedure for the above conditions and the resulting operating characteristics must be 
determined. 


Reference to Table III of MIL-STD-105B shows that for a lot size of 5,000 items 
and for Inspection Level II, Sample Size Code Letter M is designated for ordinary 
inspection. Reference is next to Table IV-A of the Stangard, the master table for 
normal single-sampling inspection. Here it will be found that for Sample Size Code 
Letter M and for an Acceptable Quality Level of 1.5% the sample size is 225 items, 
the acceptance number 8 items, and the sy number 9. ‘The acceptance-rejection 
procedure will thus be the following: (a) draw at random from the submitted lot a 
sample of 225 items and place them on life test for 200 hours, (b) determine the 
number of items that have failed by the end of this test period, (c) if the number 
failing is 8 or less, accept the lot. If the number is 9 or more, reject it. 


The operating characteristics of this plan can be determined from information 
included in the Tables of (t/u) x 100 Ratios included as part of this report. For 
this example reference will be to Table 1-D, the table of ratios for B = 1-1/3. 


‘ 
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Examination of the two lines of Acceptable Quality Level values across the top of 
this table shows that for an Acceptable Quality Level in terms of p' (%) of 1.5, the 
Acceptable Quality Level in terms of (t/u) x 100 is 4.69. With this latter ratio, 
and with the value for t, the test period length of 200 hours, the value for p, the 
mean item life for the lot, can be determined. Thus: 


(t/u) x 100 = 4.69 (ag) 
(200/u) x 100 = 4.69 
pw = 4,260 hours. 


One now knows that the operation of the plan is such that if the mean item life for 

the lot is 4,260 hours or more the probability that it will be accepted is high. (A 

rough value for this probability may be found from the operating characteristic : 

curves in the Military Standard. For an AQ of 1.5 and for Code Letter M one may 

ene the probability is approximately .97.) The Acceptable Quality Level is thus 
»260 hours. 


The ability of the plan to protect the consumer may be measured by the lot mean 
life for which the probability of acceptance is low. The tables of (t/y) x 100 
ratios included in this report include ratios at the Lot Tolerance Percent Defective 
(LTPD) quality level, the level at which the probability of acceptance is .10. ‘These 
ratios will be found in the body of the tables. Reference to the same table, Table 
1-D for 6 = 1-1/3, gives an LTPD ratio of 13 for Sample Size Code Letter M and an 
AQ of 1.5. Computations similar to those previously made give: 


(t/u)x 100 = 13 (LTPD) 
(200/u) x 100 = 13 
p = 1,540 hours. 


One now knows if the mean life for the items in the submitted lot is 1,540 hours or 
less, the probability of it being accepted is at most .10; the probability of its 
rejection is at least .90. 


The operating characteristics in hours as computed above apply also (with the 
same values) if a double-sampling or a miltiple-sampling plan for the same Sample 
Size Code Letter and AQ value is employed instead of single-sampling plan. For 
{ double-sampling in this application, the data for the plan will be found in Table 
IV-B of the Military Standard. The first sample size will be 150 items. These 
sample items would be tested for 200 hours. If 5 or fewer items failed within this 
time, the lot would be accepted; if 14 or more failed, it would be rejected. If from 
6 to 13 failed, a second sample of 300 items would be selected and tested for 200 
hours. If the total mmber fe’ ing (in the first and second samples combined) is 13 
or less, the lot would be accepted, if it is 14 or more, the lot would be rejected. 


| 

Example (2) | 
For a second example, consider an acceptance-inspection by sampling application 

for which the following achievements are desired: (a) If the mean item life for the 
lot is 20,000 hours or more the probability of acc ce will be high. Lots of this 
mean life or greater are considered "acceptable." (b) If the mean item life for the 
lot is 6,000 hours or less, the probability of acceptance will be low - .10. A test 
period of 500 hours wiil be employed. It is expected that the item life distribution 


will be of the Weibull form with a value for 6, the shape parameter of 3/4. The 
value for the location parameter, 7, is zero. 


The (t/p) x 100 ratio at the A@ will be 
(500/20,000) x 100 or 2.5. 
The (t/u) x 100 ratio at the LTPD will be 
(500/6,000) x 100 or 8.3. 


iy 
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With these values, Table 1-B giving the (t/u) x 100 ratios for 6 = 3/4 may be scanned 
to determine the appropriate MIL-STD-105B plan. One note in this table that an 
AQ of 6.5 (in percent defective) corresponds to a (t/ny x 100 ratio of 2.29. This 
is the closest value available for the desired ratio value of 2.5. The colum under 
the 6.5 AQ value heading may next be scanned to find a close approximation to the 
desired value of 8.3 for the LTPD. A value, 8.2 is found corresponding to Sample 
Size Letter K. 


Thus any MIL-STD-105B plan with Sample Size Code Letter K and for an Acceptable 
Quality Level of 6.5 will give approximately the desired operating characteristics 
for the specified test period of 500 hours. For single sampling, for example, the 
sample size will be 110 items and the acceptance number 12 as indicated in the MIL- 
STD-105B tables. 


Example (3) 


The procedure to be followed for cases in which the Weibull location parameter, 
7, is not zero but is of some other known value may be illustrated by outlining a 
third example. The method to be followed in this case is to simply subtract the value 
for 7 from the value for t, the test time to get ty» and from p to get Bos These 


transformed values t, and 1, are then used for all (t/n) x 100 computations. The 
solution obtained in terms of t be and H, can then be converted back to original values 
by simply adding the value for 7 to each. 


Consider, for example, an application for which a single-sampling plan with n 
equal to 35 and c equal to 1 has been specified. This corresponds to a plan with 
Sample Size Letter H and an AQ@ (in terms of p') of 1.5% in the 105B collection. 
Item life is measured in terms of cycles of operation. Protection against lots for 
which the average item life is less than 5,000 cycles is required. From experience 
with this product it has been determined that the Weibull distribution applies and 
that a value for 7 of 2,000 cycles and a value for B of 2 can be expected. ‘The pro- 
blem is to determine a test time, t, in cycles that will enable the plan to meet the 
above requirement for consumer's protection. A related problem is to find whether 
the plan so determined will give adequate producer protection. It has been deter- 
mined that a mean item life of 16,000 can r bly be produced by a competent 
supplier. 


The first step toward a solution is to convert the required lot mean life, yu, to 
a transformed value, Ho: This new value, By» is  - 7 or 5,000 - 2,000 which is 


3,000 cycles. Next, from Table 1-F which gives conversion ratios for use when 6 = 2, 
one finds that for Sample Size Letter H and an A@ in p' (%) of 1.5, the ratio at the 
LTPD Quality Level is 38 and at the A@ it is 13.87. Since the plan is to be deter- 
mined in terms of consumer's needs, the next step is to use the LTPD ratio to deter- 
mine t,. Thus (t/u,) x 100 = 36 or (t,/3,000) x 100 = 38. From this it is 


determined that t, must equal 1140. By adding the value for 7 (which is 2,000) to 


this latter figure, the required test time in absolute terms, t, of 3,140 cycles is 
obtained. 


The related question of reasonableness of this plan for the producer may be 
answered by substitution of the test time just determined in the ratio for the AQ@. 
The relationship is that (t,/u,) x 100 = 13.87 or that (1140/n,) x 100 = 13.87. From 


this a value for HB, of 8,240 cycles is obtained. This is then converted to original 


terms by the addition of the value for 7 of 2,000 cycles. This gives an Acceptable 
Quality Level of 10,240 cycles. This is well below the level considered reasonable 
so no hardship will be imposed on the supplier. 


Note: The recent Department of Defense publication x-108(20) contains related plans 
based on the exponential distribution (Weibull shape parameter equal to unity) 
and using variables inspection instead of attributes. Plans have been made 
by the authors of this report to construct a set of variables sampling plans 

based on the Weibull distribution which should be available shortly. 
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APPENDIX - THE RELATIONSHIP BETWEEN p' AND t/y UNDER THE WEIBULL MODEL 


(a) i material in this appendix can be found in expanded form in Reference No. 
2). 


The probability of an item failing prior to some specified time x is given by 


the cumulative distribution. If the lifelength, X has a Weibull distribution, this 
probability is, 


P(X sx) for x 27; ,B >0. 
= 0, otherwise. 


Here n (sometimes referred to as the characteristic life) is the scale parameter; f, 
the shape parameter and 7, the location parameter. 


The mean (in excess of ¥) of the Weibull distribution is, 


mM 


If p' represents the probability of an item failing prior to some test time t (in 
excess of 7), then, 


1° 
p'=l-e =l-e 


[-1n(1-p") 2/8 
+1) 


Ele 


which establishes the relationship between p' and t/ under the Weibull Model. 
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Table of (t/y)x100 Ratios for Bp = 1/2 
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TABLE 1-C 


Table of (t/u) x 100 Ratios for p = 1 


1961 ASQC CONVENTION TRANSACTIONS 


Acceptable Quality Level (AQL) 


at 
t+ nom won 
al 
ala 
w HEM AD 
a 
. ~ oN 
= 
An row Hs 
—- 
Aro ont MM 
~ Fam an 
w 
AA 


-40 


p'(h) | 0.015 | 0.035 | 0.065 0.10 | 0.15 | 0.25 0.40 | 0.65; 1.0 {1.5/2.5 


w 
N NNA 
4 
ada 
N ac 
w 
: 
w 
> 
: 
w 
8 Ro 
38 = (¥)a] 
TeaeyT 38 OOT (11/2) 


499328] 
eTdures 


AM “MH AD 


254 
| 
| AA AAA da 
ia». 


aa 


829 394 


2 224 


PHILADELPHIA, PENNSYLVANIA 


Sy € 


ot 


(Ty) eTqeqdeooy 


€/T T= COTX(11/4) Jo aTQuY, 


Wns 


~ 
"=H ao 


OD 


= (V)d) AgtTend 38 


ra 
» 


~ 


eTdues 


299991 


255 
} 
| 
| KRY PAR ARR BE 
FPR RRQ NOL AQ 
|8 
BAA OF 
ola ~- OOW AM 
wo 
aid 
° | 


1961 ASQC CONVENTION TRANSACTIONS 


SE ge Te LT 19°6 | ag 69 | > ie) 
ks of €z 6T ST €t TT} | Jal org > d 
t 
| | | te | Lt | st] ft] ier] so] | H 0 
ay lz 6T | 9T fT} €T] | t N 
t i 4 t 
dy | | te | | te | Ot |] at] stl ex | 6 | 1 
6£ 2 ge | Te 4 Tt t i 
1S SE 62 lz | qT} ¢ 
t t 
SS | gt nE | 62 | 42 gt I 
29 2s of ft < H 
99 2g t + j 

sL | €9 | 1S 1 | 9 a 
28 $9 t 4 t 
t Ls t a 
i ol H i | j 
5 v 
ooT x @ 

To’ 62 | St" 22} 10°6 | | | | 46° (4/9) 
oot] OH | S'S] OTT] S9°0] S90°0 | Sto’ OF 4 

(Tov) Tease] eTqeydeooy 


€/2 T= sotzey OOTX(N/4) Jo 
a-T 


256 

| 

| | 
| | | 
| 

| | 


257 


PHILADELPHIA, PENNSYLVANIA 


€n SE ge 6T Lt ST €T oT is) 

Sh ge Te 92 22 6T Lt | St | €t | at TT] S°6 | d 
t > 

On gz "2 Te gt | | St | at 2t oT § 0 

0s ft SE of 92 gt LT ST €T t ot N 

2s try Le Te lz Tz 6T ST 

SS Ly gz 92 | Te gt qT 4 1 

1s ly | 6€ Te 92 se | te | t i ot] 

t H 

69 0S nh ; 62 t < H 

| | 2 | & ne | 9 

? i 

88 $9 t a 
t i 1 j j 

j LL i 
66 : 

Tet | ? v 

oot x! 

£9°9€ | | 6L:22 | S6"it | | | | LS°€ | | | | (11/9) 

| OH | SS] St | OT | St-0 oto $90°0 | | (4),.4 & 


d-T 


A 
| 
| 
3 


1961 ASQC CONVENTION TRANSACTIONS 


29 SS th 6£ €€ of 92 92 42 22 Tz 6T ~ is) 
"9 1s TS on TH gt Te 62 lz Se = d 
~ 
$9 6S €¢ gh Th ge te 62 lz ' ° 
4S ah 6£ ge se oe 92 N 
69 £9 Ls TS Ly TH 6€ 9€ 92 A 
a | 9 | | 4S | | tm | ‘ 1 
€L 6S Ss TS gh St TH se A 
99 29 Ls 4S 0s 6€ A 
ig 
el SS 19 1s 2s i th I 
99 €9 Ls 7 6h A 
sg 9L ol £9 nS i 
v ; 
| | €9 4 
| 
oot x 
€1°9S | | 69° 2 | G6°9E | QO" TE | ge | | Le-te | gt | Cont | | (4/9) 
oot] | on | St | ot | $9°0 | | | | | S90°0 | Sto'o} ($).4 


(Tov) TeseT etqeqdeooy 


€/T € sotgey Jo 


258 
= 
| 


< 
z 

2 | TE*QT; ET'ET! SS°6 | LL*O gt’s 1g°t 0g*2 Te’ 2/T-T 
a 21°02 | T6°9 | 69°% os*e | €L°T €g° 19° ge" €/T-T 
< Teg 19°S | Gt] 6° 6S" gt 4/T-T 
2L°9 TS*T | S9° On" se’ | oT’ ¢-0T*S9 ¢-OT%ST T 


oot | | | Ste | St | ot | S90] Seo] Sto! oto! | | 
(%) - etqeqdeooy g 


sotzey OOT x (11/9) JO 


259 
4 
A 
4 
lia 
| | 
| 
| 
» 


f 


WHICH ROAD TO SATELLITE RELIABILITY 


Richard H. Myers, Manager, Reliability 
a Hughes Aircraft Company, Culver City, California 


INTRODUCTION 
Before getting into satellite reliability specifics, let's take a brief look at 
what makes reliability so difficult to achieve in reasonable periods of time. The 
problem arises in trying to achieve, by traditional tests and fix means alone, the . 
reliability levels needed for today's complex systems. 


The following figures illustrate these key points. 


Figure 1, showing reliability level versus calendar time, is the well-known reli- 
ability growth curve. The reliability level rises rapidly at first, then more slowly 
and levels off late in development. The picture is essentially the same during produc- 
tion as manufacturing and field experience get incorporated into the design. It is 
important to recognize that regardless of how numerous and meaningful these fixes are 
there is a definite upper limit beyond which a given system design cannot be pushed. 


This fact of reliability life is illustrated more specifically in Figure 2 which 
reflects a breakdown of failures at a point in time after System B was about 18° months 
in the field. 


As noted, the top problem (which occurred cnly about once per 200 missions) con- 
tributes but 4.7 percent of all failures reported. In other words, if a 50-percent fix 
on this top problem were effected, the mean time between failure on the system only goes 
up about 2 percent. Looking at it in total, if a 50-percent fix on the top 69 problems 
were achieved, the system failure rate goes down only 28 percent. 


This doesn't mean a 28-percent reduction in failures isn't meaningful to the user 
in terms of increased mission success rates and reduced maintenance. What it does mean 
is that many infrequent types of failures make reliability improvement of complex sys- 
tems a slow expensive process. 


The above examples (typical of today's airborne, missile or ground systems) reflect 
the reliability consequences of: 


(1) increased complexity, 
(2) unknown environments, 
(3) compressed development and production schedules. 


When you compound the above traditional reliability problems by 


(4) limited numbers of high cost test prototypes, 
(5) little (if any) orbital failure feedback, 

(6) no orbital maintenance capability, 

(7) multimillion dollar costs per launch, 


the dominant role reliability assumes in satellite systems becomes apparent. 


The remainder of the report has been divided into 3 parts. The first analyzes 
satellite reliability requirements in terms of both economic need and difficulty of 
achievement. The second part outlines technical and program reliability consideratiuns. 
Described are both the technical reliability approach (required to assure that an ade- 
quate reliability level is designed into the satellite equipment) and the related reli- 
ability program plan (the various tasks and procedures necessary to implement the 
technical reliability approach). The third briefly summarizes the more important 
findings and conclusions. 


SATELLITE RELIABILITY REQUIREMENTS 
The amount of time, money, facilities and manpower invested in reliability (as any 
other key segment of development) can and should relate directly to the need versus the 
technical difficulty of achieving the levels required. 
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RELIABILITY 
Maximum For The Design 
Years 
Figure 1 -- Reliability growth versus time. 
Problem % of 
Rank Total Failures 

1 4.7 

2 4.3 

3 3.0 

+ 2.9 

5 1.9 

6 1.9 

7 1.8 

8 1.8 

9 1.6 
First 9 problems contribute 23.9 
First 69 problems contribute 55.8 
Remainder total 44.2 
100.0 


Top 69 problems = 55.8% of all Failures 
50% Improvement = 27.9% Reduction in System Failure Rate 


Figure 2 -- System B reliability analysis. 
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Let us look at each of these elements in turn. First the need, that is to say, 
what reliability does the user require? 


Granting the importance of satellite failures to critical military missions,* the 
overriding satellite reliability consideration is basically one of economics. By this 
we mean that unless exceptionally high levels of reliability are achieved, the costs of 
sustaining satellite systems becomes prohibitive. Figure 3, plotting satellite reli- 
ability versus first year costs illustrates this key fact. Various levels of one-year 
satellite reliability are shown in both percent and the more familiar MTBF (mean time 
between failures). The first year costs shown reflect initial launching costs plus the 


costs of replacing failed satellites. The reader is referred to Reference (1), 


To illustrate the economics of satellite reliability let's apply Figure 3 toa 
typical system. For example, a communication relay system in which four orbiting satel- 
lites are required to actively receive and transmit messages between a series of ground 
stations and/or aircraft. 


As established by Reference (2) and shown in Figure 3, costs per launch are moder- 
ately fixed at $6,000,000. 


Launching missile and associated orbiting guidance computer are each expected to 
achieve an 80-percent reliability. The probability therefore of a satellite being 
successfully orbited is .80 x .80 or .64, a figure which betters the 1959-1960 results 


listed in Reference (3), 


Fixing the above values permits us to then compare the first-year costs for a range 
of satellite reliability levels. To illustrate, let's cost out 40,000 and 200 hours 
MTEF reliability levels for satellites that have been successfully launched and orbited. 


For 40,000 hours MTBF satellite reliability, Figure 3 shows that first-year costs 
total a tidy $45.6 million. 


Since $45.6 million is not an insignificant sum, 40.000 hours MTBF as a minimum 
customer need does not appear unreasonable. 


What happens if actual satellite reliability turns out to be only 200 hours between 
failures? This time, based on Figure 3, we see the system costs skyrocket to about 
$1.7 billion annually. 


Although sustaining costs of this size would be prohibitive, the related 200 hours 
mean time between failure is not a pessimistic figure. On the contrary it is one of the 


best reliability levels reported to date(*) for airborne equipment of comparable com- 
plexity and functions. 


In other words, after looking at the reliability need in terms of operating costs, 
we are now seeing the other side of the coin, the technical difficulty of achieving the 
reliability levels required. Indicated is a reliability gap of about 200 times between 
current airborne communications equipment and the reliability required of essentially 
the same equipment, circuits, ana functions enclosed in an orbiting, non-maintainable 
satellite. 


Expressed in another way, the $1.6 billion first-year cost differential provides 
strong justification for planning, funding, and implementing comprehensive satellite 
development reliability programs. 


TECHNICAL AND PROGRAM RELIABILITY CONSIDERATIONS 
Previous sections of this report have spotlighted the serious problem of reliability 
in satellite systems. Shown was a gap of 200 to 1 between the best parts available today 
and the reliability levels needed to make satellites economically, as well as techni- 
cally, feasible. 


Traditional test and fix means alone just will not bridge this gap. Required in- 
stead (during the critical design and development phases) is strong reliability emphasis 
in two primary areas. 


* Such as ICBM early warning. 
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Figure 3 -- Operating costs versus satellite reliability levels. 
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First, a documented technical reliability approach that assures an adequate level 
of reliability is inherent in the basic design. 


Second, a complementary reliability program plan that provides the means for im- 
plementing the technical reliability approach. 


Before looking at each area further it is important to recognize that no one method, 
process, or control is a reliability cure-all. A good technical approach does not re- 
Place the need for complementary reliability program effort. For example, if any one 
part type in the satellite is not capable of withstanding the launch vibration, the 
redundant use of such parts by the designer would not help since all such redundant 
parts would fail concurrently and the satellite reliability would drop to zero. 


Technical Reliability Approach 

Because of the severe reliability requirements of satellite systems, specific 
development reliability ground rules must be formalized (on a project basis) before and 
not after design has begun. 


Of particular importance (because they collectively control the maximum reliability 
level of a given design) are the 5 complementary areas covering reliability requirements, 
reliability prediction, redundancy applications, circuit design ground rules, and me- 
chanical and environmental factors. Let's look at each in further detail. 


Reliability Requirements--System analysis and tradeoff studies must first be conducted 
in order to establish design objectives that define a satellite system of maximum value 
at least total cost to the customer. In such studies, reliability must be treated quan- 
titatively in order to permit realistic performance, weight, volume and total cost 
tradeoffs. 


Once established, the overall satellite reliability requirements for launch, trans- 
ition and orbit must then be allocated to subsystems and units. The requirements estab- 
lished for each phase and equipment level must be realistic, timely, and revised as 
necessary on the basis of initial and continuing reliability predictions or test results. 


The end result (as shown in Figure 4) is to provide satellite reliability require- 
ments in terms (such as mean time between failure) and at levels (subsystems and units) 
which are meaningful to responsible system and unit engineers. 


Reliability Predictions--Preliminary and subsequent estimates of the reliability con- 
tributions of random part failures are required to establish the basis for system, sub- 
system, and unit design tradeoffs. The prediction methods used will vary depending upon 
factors such as the phase of development or new versus modified circuits. They can be 
based either on a synthesis of unit or subsystem reliability levels (from similar sys- 


tems) or buildups of expected part failure rates times complexity!>), Since thermal and 
electrical stress levels are major reliability factors, they should be introduced into 
the prediction picture as soon as this information becomes available or can be estimated. 


If during the course of the sateliite development program any of these estimates 
fall short of the required reliability levels, improvement must be achieved by increased 
cooling, greater derating, more reliable parts, and/or judicious use of redundancy. 

Since previous sections have pointed out the wide gap between the reliability need 
and the best parts available today, it is apparent all of the above must be a part of any 
satellite system reliability effort. 


The application of redundancy will be considered next. Circuit design, mechanical, 
and thermal aspects are covered in subsequent sections. 


Redundancy Applications--Basic to achieving the high levels of reliability required of 
satellite systems is the judicious application of redundancy. 


This might be on a part, circuit, or equipment level or a functional or sequential 
basis. By functional we mean that all redundant parts, (©) circuits, and/or equipments 


are operating continuously, while in the sequential case(7) another element gets turned 
on by remote or other means as the preceding one fails. 
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Since the use of redundancy (as all design techniques) has many pros and cons, 
every application must be thoroughly evaluated from both the mathematical and engineer- 


ing viewpoints. (8) Some key considerations are listed below. 


(1) What are the relative advantages of functional versus sequential redundancy 
for different part or unit life distributions? 


(2) Reliability gains through part or circuit redundancy may be offset by in- 
ability to detect open connections not resulting in equipment failures. 


(3) When part redundancy is applied to protect against catastrophic failures, what 
increased probability of degradation failure can be expected? ' 


(4) Increased life testing and associated costs are required where part redundancy 
is used since part failures do not result in circuit failures. 


(5) Weight, volume, and power disadvantages relative to reliability and cast gains 
should be compared for various types and levels of redundancy. 


(6) Switching or storage life reliability effects associated with using sequential 
redundancy should be explored. 


It is important to note that any one of the above factors could be controlling in a 
given satellite system. For example, where system performance requires the use of 
“short life"* power tubes, sequential rather than functional redundancy must be used to 
effect any large gain ir satellite reliability. 


Summing up, regardless of the hue and cry about increased weight and volume, redun- 


dancy,. as shown in Reference (8) | in the only design means capable of providing the 
equivalent of order of magnitude improvements in part reliability levels. Satellite 
reliability needs, therefore, justify and require considerable analytical and engineer- 
ing effort in this key area. 


Circuit Design Ground Rules--Formalized circuit design ground rules are required to 
eliminate or reduce actual or possible modes of failure prior to and not after the cir- 
cuits have been finalized. Specifically, satellite design and development programs 
should include at least the following reliability ground rules. 


(1) Maximum use (at the initial design stages) of parts qualified for expected 
environments and supplied by approved vendors. 


(2) Utilization of all parts at 25 percent or less of rating to reduce random 
failures due to electrical stress. 


(3) Utilization of all parts well within thermal ratings to minimize random fail- 
ure and chances of exceeding part temperature limits. 


(4) Marginal checking 9) of all circuits to minimize probability of degradation 
failures due to changes in part parameters with time. 


(5) Maximum use of silicon rather than germanium semiconductors to minimize launch 
and boost thermal shock failures. Use of silicon devices also reduces possibilities of 
part temperature limits being exceeded during checkout tests prior to launch. 


(6) Elimination of parts having short average life times and known wear-out char- 


acteristics (such as relays and power tubes‘ 29)» 

(7) Where redundancy has been applied at the part leve1, (©) detailed circuit 
analyses are essential to assure that whether a redundant part opens or shorts the cir- 
cuit will still function. 


As reflected in the above check list, exceptionally conservative circuit design 
practices are required in order to help meet satellite reliability requirements. Since 
backtracking is extremely difficult and expensive, they must be formalized for each 
satellite program before and not after design has begun. 


* Relative to mission length, e.g., 1000 hours relative to a year satellite reliability 
requirement. 
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Mechanical and Environmental Factors--Major mechanical and environmental factors af- 
fecting satellite reliability are listed below. 


(1) A heat transfer system that assures minimum part temperatures. 
(2) Use of redundant joints and connections. 
(3) Adequate environmental protection. 


Let us examine each of the factors further. Because of the severe satellite reli- 
ability requirements and the direct relationship between part failure rates and ambient 
temperature, parts must be held at minimum (rather than traditional maximum) tempera- 
tures. Reliability predictions for typical systems clearly indicate that ambient tem- 
peratures in the range 25 degrees to 35 degrees C are required to provide part reli- 
ability levels consistent with unit and system needs. 


Another basic reliability factor is the use of redundant joints and connections. 
The objective here is to achieve a low collective probability of failure for the thou- 
sands of joints and interconnections required in fabricating and assembling satellite 
systems. Although the use of redundant joints and connections is recommended practice 
in all types of satellite equipment, it becomes essential where redundancy has been 
applied at the part level. Without redundant joints and connections much of the gain 
resulting from part redundancy would be lost through the increased Joint and connection 
failures. 


As regards environmental capabilities, realistic design margins of safety should be 
incorporated to allow for environmental extremes (not averages) in the areas of vibra- 
tion, shock, acceleration, etc. Of additional consequence are means of environmental 
protection relative to long term satellite systems exposure to solar radiation, micro- 
meteorites and the vacuum of space. 


Previous paragraphs have outlined the primary technical reliability requirements 
needed to assure that an adequate reliability level is inherent in the basic satellite 
design. 


The next part describes the complementary reliability program plan, in other words 
the various tasks and procedures necessary to implement the technical reliability 
approach. 


Reliability Program Plan 
A satellite reliability program plan is best viewed in terms of the types of fail- 


ures that must be predicted, prevented, detected, and corrected during the critical 
design and development period. 


Specifically, the following failure contributors are recognized, the summation of 
which will effectively determine the reliability levels achieved by operational satel- 
lites. 


1. Random part failures. 
2. Degradation failures. 
3. Part and unit environmental failures. 
4, Pailures caused by: 
a. Packaging inadequacies. 
b. Fabrication processes. 
c. Poor workmanship. 
a. Checkout procedures. 
e. Storing and shipping practices. 


Consistent with the severe reliability needs of satellite systems, this section has 
been divided into 9 parts covering reliability organization, time phased reliability 
programs, reliability design reviews, parts reliability, design and qualification test- 
ing, manufacturing reliability assurance, supplier reliability controls, failure re- 
porting and analysis, and reliability training. 


In effect we have listed above those major areas of reliability effort which are 
basic to any effective reliability program (satellite or otherwise). 


The key difference is that many steps in the reliability program for a non- 
mainteinable satellite are essentially irreversible. It is like a tightrope walker who 
makes 99 good steps (out of a possible 100) and one misstep. From the viewpoint of the 
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audience he tried his best and did 99 percent of the job. Taking a more practical look 
at it, he had zero reliability. 


Reliability Organization--Satellite reliability program needs require the full-time 
directed effort of reliability specialists. This reliability group can be drawn from 
a functional reliability organization and report to the project manager responsible for 


the entire satellite program requirements. Major responsibilities(22) of the project 
reliability group and supporting specialists would include, but not be limited to, the 
following. 


Advisory or 


In-Line Liaison 


Environmental Conditions Determination 
Reliability Apportionment 

Reliability Indoctrination 

Parts Approval Verification 
Specifications Review 

Design Review 

Failure Reporting Surveillance 
Statistical Test Planning 

Statistical Test Evaluation 

Human Factors 

Quality Control Coordination 

Program Data Evaluation 

Vendor Control x 
Flight Test Planning 

Analysis of Test and Flight Failures 4 
Determination of Corrective Action x 
Corrective Action Followup xX 


Fw 


Time Phased Reliability Programming--Time phased reliability programs are required for 
each of the satellite subsystems. These schedules recognize that reliability is a 
parameter that can be quantitatively specified, estimated, assessed and measured at 
predesignated times in a system's design and development. A specific satellite reli- 
ability program is shown in Figure 5. As outlined, the reliability program has been 
subdivided into a number of specific reliability tasks and related tests. Start and 
completion dates are also blocked out to permit measurement of progress as the program 
proceeds. The completion dates are derived from, and consistent with, the master pro- 
gram schedule. 


Reliability Design Reviews--A system of reliability design reviews permits evaluation of 
design reliability at sequential development stages. The objective here is the timely 
detection and correction of actual or potential modes of failure prior to and not after 
the design has been finalized. 


Reviews (depending on the development stage) will include reliability evaluation of: 


1) Predicted versus required unit and system reliability levels (Figure 6) 
Redundancy applications 

3) Consequences of opens versus shorts in redundant part circuits 

4) Selection and use of approved parts 

5) Parts used well within electrical and thermal ratings 

6) Degradation failure calculations 

7) Design and qualification test results and related corrective action. 


Parts Reliability--Because the part random failure rates required to meet the satellite 
system reliability levels must be significantly lower than industry rates on comparable 
equipments, heavy parts reliability effort and support is justified. 


Parts reliability program areas considered essential to meeting this major reli- 
ability objective are listed below. 


(1) Development by components groups and use by designers of qualified purts lists. 

(2) Development by components groups, use by designers and purchasing of approved 
vendor lists. 

(3) Complete qualification of non-standard parts prior to final design release and 
orbital flight tests. 
(4) Comprehensive analyses of all part failures. 
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Figure 6 -- Reliability prediction worksheet 
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(5) Specialized parts tests in the following areas: 


(a) Non-destructive tests (e.g., infrared hot spot checks of resistors) to 
cull out potential failures. 

(b) Screening of parts parameters to minimize circuit drift failures. (12) 

(c) "Burn-in" of parts (such as diodes) to detect early part failures prior 
to assembly and to improve failure rates. 


(6) Comparative vendor tests on high use parts. Significant reliability gains can 


be expected in this area since experience(10) has shown failure rates can vary by 25 to 
1 for the same part supplied by different vendors. 


Design and Qualification Testing--Regardless of how valuable the preceding reliability 
techniques and controls prove to be, functional, environmental and reliability testing 
will still be necessary. 


The objective here is the timely evaluation and qualification of parts, units and 
systems prior to high cost orbital flight tests. Five sequential steps are involved. 


(1) Establish functional, environmental and reliability requirements. 
(2) Schedule manpower and facilities. 

(3) Develop test procedures. 

(4) Conduct tests, analyze failures, and improve design or processes. 
(5) Approve and release qualified parts, units and systems. 


Manufacturing Reliability Assurance--Increased quality control effort is necessary on 
satellite equipments to prevent or reduce failures caused by fabrication processes, 
material and part inadequacies or poor workmanship. In brief, reliability must be built 
as well as designed into satellite systems. Required are effective quality organizations 
and procedures. Some basic quality areas are listed below. 


Calibration of test equipment, tools and gauges. 
Control of non-conforming supplies. 

Suppliers quality programs. 

Drawing and change controls. 

Records of quality conformance. 

Training and certification of personnel. 
Receiving, in line, and final inspection and test. 


NS 


In addition to increased emphasis in these basic quality areas, requirements pecul- 
iar to satellite programs include: 


(8) 100 percent not sample screening of critical parts parameters, 
(9) "Burn-in" of parts prior to assembly, 
(10) Increased 100-percent application of non-destructive test methods, 
(11) Segregation and "kid glove” handling of incoming and in process parts, mater- 
ials and assemblies, 
(12) Where part redundancy has been used, inspection and test techniques must be 
developed and applied for detecting part, joint, or connection failures within operable 
circuits. 


Supplier Reliability Controls--In order to implement satellite system reliability levels, 
it is necessary to pass on to suppliers their share of the total satellite reliability 
requirements. 


In the parts area supplier reliability controls involve 4 sequential steps. 


(1) Survey and do business with only those suppliers capable of meeting the part 
reliability requirements. 

(2) Carefully select and qualify parts before releasing for design application. 

(3) Requalify samples .from every shipped lot prior to assembly into satellite 
equipments. 

(4) Rate the part supplier on the basis of qualification test results and reli- 
ability experience with parts received. 


An effective program of subcontractor reliability control is also necessary. As 
previously outlined for parts, suppliers of major units and subsystems must be surveyed 
and rated. In addition, formalized time phased programs and reliability requirements 


and tests(13) should be established early in the subcontract development program. These 


| 
| 
| 
J 
i 


PHILADELPHIA, PENNSYLVANIA 273 


programs must be carefully monitored overtime by the cognizant design and outside pro- 
curement personnel in order to achieve timely integration of reliable subcontract equip- 
ment into the overall system. 


Failure Reporting and Analysis--A system of failure reporting and analysis is a funda- 
mental part of the laboratory, fabrication, and flight test phases of the satellite 
reliability program. 


Early in the development period procedures that assure rapid failure collection and 
analysis should be established. Also required are cross-indexing methods for purposes 
of detecting recurrent failure modes or types, regardless of circuit, unit, or subsystem 
application. 


Based on failure evaluation, supplier or internal responsibility must first be 
established and then corrective action initiated in terms of changed design, improved 
fabrication process, or a test procedure modification. Also of primary importance, due 
to multimillion dollar flight test costs, are the methods of followup that insure cor- a 
rective action has indeed been accomplished. a= 


Reliability Training--Since, as previously established, satellites require reliability 
levels several orders of magnitude better than today’s best comparable equipment, it 
becomes important that project, engineering, and manufacturing personnel be made aware 
of the reliability programs objectives, needs, and methods. 


The reliability training programs should include seminars describing the respective 
and complementary roles reliability design reviews, parts reliability, design and quali- 
fication testing, manufacturing reliability assurance, supplier reliability controls, 
failure reporting, and analysis, play in meeting the overall satellite reliability re- a 
quirements. 


SUMMARIZED CONCLUSIONS 
The more important conclusions are considered to be as follows. 


Exceptionally high levels of reliability must be achieved in satellite equipment 
because of multimillion dollar costs per launch and the replacement satellites needed 
to keep the system operational. 


A comparison of satellite reliability need (in terms of operating costs) and the 
best parts available today indicated a reliability gap of about 200 to l. 


Traditional test and fix means alone just will not bridge this gap. Required in- 
stead to provide solutions to the satellite reliability problem, is strong reliability 
emphasis in two complementary areas. 


First, a documented technical reliability approach concurrent with (not subsequent 
to) each stage of design and development. 


Second, a related reliability program plan that provides the means of implementing 
the technical reliability approach. 


In order to achieve satellite reliability levels of 40,000 hours mean time between 
failures the technical reliability approach must consider 


1) Part, circuit and/or equipment redundancy, 

2) Design application of parts well within electrical and thermal ratings, 
3) Minimum part ambient temperatures, 

4) Maximum use of qualified parts from approved vendors, 

5) 

6) 

7) 


Marginal checking of all circuits, . 
Elimination of parts with known wear-out characteristics, and 
Redundant joints and connections. 


Implementation of an effective satellite reliability program plan requires 


(1) A full-time project reliability group for technical and programming purposes, 

(2) Initial and continuing reliability comparisons of required versus predicted 
reliability, 

(3) A specialized parts test program including screening, culling and “burn-in” 
techniques, 

(4) Comparative vendor testing to establish the “best” source, 
sit (5) Complete qualification of parts, units, and systems prior to orbital flight 
ests, 
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(6) Increased quality control effort including the 100-percent application of non- 
destructive inspection and test methods, 

(7) Improved monitoring of supplier reliability programs, 

(8) Reduction of failure reporting, analysis, and correction cycle times during 
the critical design and development satellite phases, and 

(9) Training seminars to promote project awareness of the satellite reliability 
programs’ objectives, needs and methods. 


In summary, unless exceptionally high levels of reliability are achieved in satel- 
lite systems the costs of replacing failed satellites becomes prohibitive. Outlined and 
discussed were the technical and program reliability considerations needed to make sat- 
ellite systems economically as well as technically feasible. 


REFERENCES 
(1) Aroian, L.A., and Myers, R.H.,"Redundancy Considerations in Space and Satellite 
Systems ," Proceedings of the 7th National Symposium in Reliability and Quality 
Control, January 1961. 


(2) The Los Angeles Times, 3 August 1960, Part 3, p. 1, col. 5. 


(3) Herrick, J.W., Editor, "Space Log," Space Technology Laboratory, Vol. 1, No. l, 
1 July 1960, p. 58 


(4) “Reliability Training Text," Institute of Radio Engineers, 15 September 1959, 
p. 215. 


(5) “Hughes Designers Reliability Handbook,” Hughes Aircraft Company Report R-67, 
27 February 1959. 


(6) Creveling, C.J., "Increasing the Reliability of Electronic Equipment by the Use of 
Redundant Circuits,” NRL Report 4631, Naval Research Laboratory, Washington, D.C., 
5 December 1955. 


(7) Aroian, L.A., "Maximization of the Reliability of a Complex System by the Use of 
a in Sequences," Space Technology Laboratories Report STL/TM/60-0000-00034, 
1960. 


(8) Myers, R.H., "Mathematical and Engineering Aspects of Satellite Redundancy," Hughes 
Aircraft Company Report R-60-3, 1 September 1960. 


(9) Taylor, N.H., "Designing for Reliability," M.I.T. Lincoln Laboratory, Proceedings 
of the IRE, June 1957. 


(10) "A Selection of Electron Tube Reliability Functions,” ARINC Publication No. 110, 
8 January 1958. 


(11) "Guides fdr Reliability Organization," U.S. Air Force Specification Bulletin 
No. 510, 30 June 1959. 


(12) Gottfried, P., Schneider, L.L., and Jones, R., "Screening Tests for Component 
Reliability Improvement," Transactions of the 5th National Symposium Reliability 
and Quality Control, January 1959. 


(13) Hill, D.A., and Myers, R.H., "Procedures and Tables Based on Acceptable Reliability 
Levels," Hughes Aircraft Company Report R-58-2, 4 August 1958. 


| 

i 

| 


RELIABILITY INSTRUMENT PANEL 
Frank M. Gryna, Jr. 


THE MARTIN COMPANY 
Baltimore 3, Md. 


Continuous control of product reliability requires measurement - both of 
achieved product reliability and of the tasks in a reliability program which 
eventually determine the final reliability of the product. 


This paper first reviews the basic elements of reliability planning and 
control and then-presents a measurement technique for control - The Reliability 
Instrument Panel. 


RELIABILITY PLANNING AND CONTROL 


To achieve a required level of reliability under operational conditions, a 
reliability program must include technical tasks to be performed by a variety 
of functions-primarily design, procurement, manufacturing, test, inspection, 
specifications and field support. Some of the tasks are new (i.e. reliability 
prediction) and some of them are established tasks which must be controlled 
better (i.e. process control during manufacturing) but the number of tasks is 
large and varied. It has been my experience that a program for a complex pro- 
duct will contain at least 30 individual tasks. In addition to the technical 
tasks there is also a management task of (a) establishing the program and (b) 
assuring the proper execution of the program. I believe that the measurement 
of task performance is a weak link in the management of reliability programs. 
To put the measurement phase in proper perspective, the basic management process 
underlying the efforts to make: a reliable product should be examined. 


Juran? has stated four conditions that must be present simultaneously in 
order to achieve control of a manufacturing process. 


1. The operator must know what he is supposed to be doing. 

2. The operator must know what he is = 

3. The operator must have means for regulating the process. 

4, The operator must feel that it is to his personal advantage to regulate 
the process to achieve good quality. 


These four conditions are just as necessary for the individual technical 
tasks in a reliability program. (Furthermore, these points provide an excellent 
check list to detect underlying causes of weak areas in reliability programs). 
Here's what this means in terms of a reliability program (Figure 1 presents 
these four conditions in relation to the important distinction detween planning, 
operations and control.) 


1. The program must be defined. 


Many companies have now been sold on the need for a reliability program 
and are now in the implementation phase. The first step in implement- 
ing a program is to completely define the program. This is really the 
planning phase of a reliability program. On a compa wide basis, 
this means that reliability policies must be established and the reli- 
ability responsibilities of each organizational unit clearly defined. 
(Reference 3 shows an example of this.) Too much time is being wasted 
by reliability specialists arguing about the one best organizational 
approach. It is time that we realized that the reliability organiza- 
tion must be designed to fit within the operations of the individual 
company. The definition of responsibilities must be established and 
preferably in writing. If the reliability specialists within a 
company cannot reach agreement then management is left no alternative 
but to dictate the responsibilities. On a project basis, the individ- 
ual reliability tasks necessary for the project must be (a) defined, 
(bo) responsibility established for each task and (c) time schedules 
and budgets established. (A technique for defining the tasks in a 
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reliability program is shown in Reference 4.) Finally, standards of 
performance must be established not only for product reliability but 
also for the performance of the technical tasks in the program. Great 
strides have been made in establishing numerical standards of reliabi- 
lity performance for the product. Not only is a numerical reliability 
requirement set on a total system, but this requirement can be divided 
into sub requirements on individual subsystems thereby giving each 
design group a reliability number to design to-something they never had 
before. Making reliability a design parameter like weight is probably 
the most significant of the new reliability techniques. However, 
little has been done to apply the concept of a standard to other tasks 
in a reliability program such as qualification tests, design reviews, 
preparation of failure reports, implementation of corrective action 

for hardware failure, etc. I submit that the establishment of adequate 
standards of performance for all tasks directly or indirectly effecting 
product reliability is vital to achieving continuous control of a reli- 
ability progran. 


Performance must be measured and evaluated against the standard. 


The measurement of performance in a reliability program should consist 
of three categories: 


a. Measurement of product reliability against the reliability 
requirement. 


This is, of course, the ultimate measure of the effectiveness of 
the entire program. It should be noted that product reliability 
measurement must not only include measurement of the end item but 
measurement to as low an item level (subsystem, chassis, etc.) as 
practicable in order to (a) isolate equipment problem areas and 

(bo) collect data for use in preventing failures in future products. 


b. Measurement of performance of tasks in « reliability program 
against a standard. 


From the viewpoint of monitoring a program and preventing future 
failures, the measure of product reliability is not enough. All 
tasks that effect the final product reliability must be measured 
and controlled. The measurement of these tasks gives an indica- 
tion of the final product reliability. Thus some of the potential 
problem areas can be detected before complete systems are built. 


c. Audit of performance of tasks in a reliability program. 


The performance of some tasks in a reliability program cannot be 
measured in numerical terms. For these tasks, audits should 
periodically be made to assure that the objectives and procedures 
originally developed are still appropriate and are being followed. 
Design review activity is a case in point. 


The simple process of recording performance and feeding the information 
back to line personnel can be a powerful force in improving perform- 
ance. The quality manager of an electrical connector plant once told 
me that he could almost guarantee a quick reduction in percent defec- 
tive by posting a percent defective control chart in the production 
area.” Furthermore the improvement took place before there was enough 
data to compute control limits and before the quality manager made any 
investigation. This indicates that the improvement was due to some- 
thing much more basic than a statistical technique. 


The measurement and evaluation phase is really the contegi phase of 
the program. The distinction between the planning phase“(setting 


policiee, defining responsibilities, procedure schedules, budgets and 
standards) and the control phase (measurement and evaluation) is 
important because it emphasizes that the two phases, although tied to- 
gether by feedback, are separate and distinct. Reliability tasks de- 
fined to aid end product reliability are really a set of plans each of 
which must be accompanied by a specific means of control. 
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3. A_means for taking corrective action must be available. 


This is the operations phase of the program. In the case of product 
failure due to a design deficiency, the responsible design group must 
be provided with the time, funds, and staff help required to determine 
the cause of failure and the corrective action necessary. In the case 
of inadequate failure reports from the field, the field must be told 
the specific inadequacies and instructed in correcting them. Again, 
staff help may be required to help the line people correct the problem. 


4. Personnel must be motivated meet the standards of performance 
established for their tasks. 


On a company wide basis, reputation is a powerful force. The possibi- 
lity of incentive contacts and penalty clauses on reliability provide a 
more immediate avenue. The hanging of rejection tags on equipment 
which has passed all inspection tests but for which known reliability 
problems exists will provide even more immediate motivation. 


People inevitably underly all of the tasks in a reliability program. 
The surface has only been scratched in developing effective means of 
motivating people. Experience in the manufacturing activity shows that 
financial incentives are not the complete answer. The variety of 
personnel engaged in a reliability program make the task of motivation 
a difficult one. 


THE RELIABILITY INSTRUMENT PANEL 


The key part of the control phase is the measurement of performance cf (a) 
product reliability and (b) performance of key functional tasks in the program. 
I would like to propose a Reliability Instrument Panel > to measure and report 
this information. In essence, the R.I.P. will be a listing of results for a 
number of indices representing tasks that directly or indirectly effect product 
reliability. 


The development of a useful Reliability Instrument Panel presents a real 
challenge to the reliability engineer. Here are some critical pointe to examine. 


1. Development of adequate performance indices. 


The development of numerical indices is the heart of the problem, par- 
ticularly for reliability tasks that do not lend themselves to quanti- 
tative measurement. The following criteria for indices are critical: 


a. Ease of understanding. 


The index must be ~ i and clear. When this is not possible 

and this w so for many steps), it is better to omit an 
index entirely than invent an arbitrary and/or complicated index 
merely for the sake of having an index. Gaining acceptance of the 
basic concept of performance measurement is more important than 
having a complete set of indices. More complicated indices can be 
added later if the need is critical and preferably in answer to a 
request from the affected organizations. 


be. Ease of data collection and computation. 


The effort required to publish the report must be reasonable in 
order to (1) publish the report frequently and on a timely basis 
and (2) permit the reliability specialists to have time to execute 
their regular functions in reliability. 


ce. C lete valid for the task to be measured. 


Complete validity is essential if an R.I.P. is to be an effective 
tool of reliability control. It would indeed be tragic for any 
control system to degenerate into arguments between line and staff 
personnel on whether or not an index gives a "true" picture of per- 
formance. Again, it would be better to omit an index rather than 
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jeopardize acceptance of the basic concept by using an index that 
is only partially valid. 


Ideally, the control for each task in a program should include a 
quantitative index for evaluation. To achieve this, it is necess- 
ary to define the task in very specific terms. A criterion for a 
proper task definition should be the ability to evaluate the per- 
formance of the task and perhaps even the ability to establish a 
quantitative index of performance. 


2. Development of numerical standards of performance. 


For each index developed for the Reliability Instrument Panel, a stan- 
dard value for the index should be ertablished. A key point is this: 
the people responsible for the activity should set their own standard. 
At first glance, it might seem that this would result in very loose 
standards. Actually, this often results in a tighter standard. More 
fundamental, however, is the question of corrective action. Real 
effort to take corrective action to meet a standard is much more likely 
when a standard is self-imposed rather than established on the basis of 
higher authority. 


3. 


Decisions on any project must be made by drawing the proper balance 
between reliability, production schedules, and overall project costs. 
In the case of critical failures, there is no problem in convincing 
Management that corrective steps must be taken even if delivery delays 
and increased costs will result. However, the reliability engineer is 
often faced with selling management on a phase of the program or a hard 
ware fix that either has only a minor effect on final reliability or 
only indirectly effects reliability. If the recommendation is not 
accepted, the reliability engineer is prone to criticize management for 
its failure to appreciate the importance of the problem. 


It seems to me that the basic fault lies with the reliability engi- 
neer's failure to present the case in terms familiar to management. 
With each recommendation, must be presented the effect, in quantitative 
terms, on reliability, schedules and costs. It is not always possible 
to show numerically the effect on reliability. In these cases, it is 
only natural for management to place more weight on the schedules and 
cost factors because invariably someone is able to estimate increased 
costes and schedule delays. Now here is the key point: most recommen- 
dations made to increase reliability will result in lower total program 


short or long run. 


The real challenge to the reliability engineer is to turn the cost and 
schedule factors from being his enemies to being his tools to sell 
management. This can be done by estimating for management the dollars 
and time that is currently being lost due to the program weakness that 
the reliability engineer is trying to correct. (Projecting the current 
rate of loss through to the end of a time period or the end of a cont- 
ract can be a particularly powerful tool for convincing management. 
Computer simulation techniques as applied in PERT, for example, are a 
fascinating possibility.) 


Table 1 shows a possible format for a two page Reliability Instrument Panel 
for a specific project. The form provides for the indices, organization resp- 
onsible, a standard for each index, current performance, performance last month, 
and statements on the effect of performance on product reliability, schedules 
and costs. The indices finally selected will depend on project needs and 
company practices. The list in Table 1 is presented to provide ideas to reli- 
ability engineers throughout industry. 


CONCLUSION 


The Reliability {nstrument Panel will provide line supervision and top 
Management with a continuous picture of performance so that steps can be taken 
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early enough to prevent serious problems from developing. This, in turn, will 
help assure that reliability remains free from the concept of "management by 
drives". (One month there is a drive for lower costs, and costs are lowered, 
for a while at least. Next month, the drive is for a better safety record.) 


Lasting improvements in reliability can only be achieved when continuous control 
rather than sporadic fire-fighting is achieved. < 
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MYTHS AND REALITIES IN RELIABILITY 


C. M. Ryerson, Product Assurance Administrator 
RCA, Camden, N. J. 


INTRODUCTION 


Many existing concepts about reliability are either unrealistic or 
incomplete. There is usually an element of truth in each, but a frame- 
work of myth is frequently established around it which misleads the 
hearer or reader. Sometimes meaning of words is the basis for the myth. 
Other times conclusions are drawn on inadequate evidence or based on 
statements withdrawn from context. 


This paper considers several of the common myths in turn and dis- 
cusses some Clarifying facts. 


DISCUSSION 


Myth #1 

Reliability is not a basic military or national defense problem. 
It is just a fad that will blow over if industry sits tight and keeps on 
making its usual good products. 


- The seriousness of unreliability in complex systems is very real 
to those who have tried to improve system reliability to satisfy existing 
field needs. Quotations from responsible authorities will prove this 
point: 


Quoting from a December 10, 1959 report by Mr. J. M. Bridges, 
Director of the Office of Electronics, Defense Department, Washington, 
D. C. to the Defense Science Board: "Poor reliability is one of the most 
serious limitations in obtaining acceptable operational effectiveness in 
military systems." 


Quoting from a report by Mr. N. E. Golovin, Director of Technical 
Operations Division, Advanced Research Projects Agency, Washington, D. C. 
to the same meeting of the Defense Science Board: "The forecast of opera- 
tional feasibility for any (complex) system must be a relative one with 
respect to the integrated effects of cost and over-all system reliability 
in competitive alternatives." 


Quoting from an address by Vice-Admiral J. T. Hayward, presented to 
the 6th National Symposium on keliability and Quality Control held in 
Washington, D. C. on January 12, 1960: "The unreliability of complex 
systems and their component parts is one of our major national problems." 


Quoting from a paper by Capt. Emerson F. Fawkes, USN, Assistant Chief 
of Staff (Material) presented at the 3rd Navy-Industry Conference: "We 
in the fleet daily receive millions of @llars wrth of unreliable equip- 
ment.....its usefuiness is badly compromised." "...in spite of all ef- 
forts to improve reliability of our weapon systems, increasing complexity 
is creating an ever-increasing reliability deficit." 


Excerpting information from a paper given by the U. S. Navy Commander 
C. O. Holmquist, Office of Assistant Secretary of the Navy (R & D) at the 
3rd Annual Navy-Industry Conference, 1959: (1) Aircraft average avail- 
ability in the fleet has dropped from a near maximum (maximum of 80% 

based on Navy turn-around and preventative maintenance program) of 79.5% 
ema aircraft available (propeller type) in 1951 to about 55.6% 
average for the more complex jet types in 1959. (2) Typical complexity 

of electronic gear for fighter aircraft has risen from less than a hun- 
dred parts in 1941 to over 8,000 electronic parts in 1959. (Based on an 
approximate figure of 10 parts per tube). Reliability and maintenance are 
thus serious problems today. (3) The average maintenance figures on the 
more complex current fleet aircraft range to as high as 72 maintenance man 
hours for each flight hour. This contrasts with the less than 5 to 1 
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ratio common just a few years ago. 


From this it can be seen that reliability is a major national prob- 
lem and not a fad that will blow over. 


Myth #2 
There is no real problem in making most modern systems reliable. 


Consider, for example, the automobile or the average television set. 


- Let us consider the automobile. The average family car is driven 
10,000 miles a year and is replaced before four years of use as worn out. 
This means the average age of the family car is 40,000 miles or at 
a 30 mph average, 1,330 hours of operating time. Suppose that in this 
four years you replace two sets of tires, two fan belts, five sets of 8 
spark plugs, a battery, three sets of points, one set of brake linings, 
and two essential motor parts such as a fuel pump. With no more repairs 
than this in four years the average family would consider their car very 
reliable. Yet even considering each set of parts such as 4 tires, 8 
spark plugs, etc., as one failure event, this story includes 12 failure 
events. The mean time between failures for this "reliable" family car is 
thus only (1,330/12) 110 hours MTBF. 


Now if all the failed parts are considered as individual items, such 
as is frequently done in military field reporting programs, the total mean 
time between failures for this "reliable" car would appear to be 1,330/60 
or 22 hours. 


We all are familiar with the unfortunate cases of cars which are pro- 
duction "lemons". Such cars may exhibit five to ten times as many fail- 
ures as those enumerated for the typical “reliable" car. The mean time 
between failures for these on the total failed item basis would approach 
a 2 hour figure. 


We thus see that the so-called reliable family car may exhibit a 
MTBF of between 2 to 125 hours. On this basis the military requirement 
for equipment with a MTBF of from 500 to several thousand hours is a se- 
vere requirement. The reliability of complex systems is a real problem. 


Myth #3 
The average company can accomplish all that is necessary in relia- 
bility without any special program. 


- The average company operates now as it has in the past with a so- 
called good practices approach. This must be replaced by a controlled 
operation if reliable complex systems are to be produced. 


The differences between the good practices approach and a controlled 
operation and how to achieve the conversion are subjects for other papers. 
Indeed, the "how to" portion of this is today one of the major problems 
of industrial management. 


In brief for this presentation, these are changing times. Products 
are changing, techniques are changing, and organizational approach must 
change to keep pace. A formal program for reliability control and product 
assurance is a must for the success of all future projects involving com- 
plex systems, A brochure on the RCA Product Assurance Program to meet 
this need is available on request. 


Myth #4 
If you try to measure the reliability of your product you just run 
the cost up to where you are out of business. 


- In the near future the only companies left in the business will be 
those who know and are able to control the reliability of their product. 
The first step on this road to survival is to establish methods and pro- 
cedures for reliability measurement. Rather than being a matter of not 
being able to afford reli ability measurement, the situation is rapidly 
becoming one in which no company can afford not to make them. Reasons 
for this will be discussed in connection with other myths. 
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Myth #5 

It is impossible to design and develop a new system to a specified 
numerical reliability figure. Reliability must grow as experience is 
gained from use. 


- There are two aspects to discuss in connection with this myth: 


(1) It is now possible to design new systems to a specified numerical 
reliability figure. There is less information and empirical data to use 
as guidelines on some types of systems than others, but the basic techni- 
ques and controls are available. Multiple sequential feedback of analyzed 
information and control in specific stages with design reviews are some 
of the modern tools of controlled operation. 


(2) Those who depend on a reliability growth as a product progresses 
through various stages of manufacture and use are using the god prac- 
tices approach. This is the expensive and outmoded method whereby cor- 
rective action is after-the-fact and the major effort is expended in 
“putting out fires" as they spring up. The economy of the opposite ap- 
proach or controlled operation derives from taking deliberate advance 
measures to prevent "fires" from occurring. This requires a team analyt- 
ical approach for doing the best things in the right way at the right 
time. 


The best and least expensive equipment is that which was designed 
right the first time with careful consideration of all the end-use factors, 
human limitations, and production problems. 


Myth #6 
Good company practices of yesterday will achieve reliability today 
if strictly adhered to. 


- A simple answer to this is that the good practices of today and 
tomorrow are not the same as the good practices of yesterday. It is un- 
fortunately true that many known poor practices of yesterday are still 
being practiced in parts of industry today. These must first be elimi- 
nated and replaced with a controlled operation based on the new good 
practices which are quite different. 


Myth #7 

Presently available parts are good enough to meet any reliability 
requirement if the circuits are designed properly for optimum application 
of the parts. 


- It is true that many reliability failures in existing equipment 
can be traced to the misapplication of parts. Overloading and bad en- 
vironment can make an otherwise good part perform unreliably, But the 
limiting factor today in the reliability of complex systems is that even 
with optimum application and maximum derating available, parts simply are 
not good enough. 


Consider the former illustration of the family automobile. Most 
modern cars make the best possible use of available parts. To greatly 
extend car reliability would require the development of better parts such 
as better batteries, better tires, etc. 


Myth #8 
We have an experienced procurement operation and can get myies 
y 


we need. If "better parts" is the answer, all we have to do is speci 
them to our suppliers. 


- This raises the question of a real dilemma which exists in modern 
procurement. Do you specify to your suppliers the exact high reliability 
you know you need when this is above the state-of-the-art and you know 
you will have to waive the requirements in order to get anything at all, 
or do you order what you know you can get, knowing full well that this 
will not result in the system reliability you need. There is no pat ans- 
wer to this dilemma nor can there be until the problem is solved by the 
availability of better parts with measured better reliability. 
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Myth #9 
You will get a reliable product if you design to use only mil stand- 
ard parts. 


- This is a complete fallacy because of the wide range of relia- 
bility that exists in presently available mil standard parts. Even with 
the best application and maximum possible derating, many available mil 
standard parts have very poor reliability. The existing part specifica- 
tions are inadequate to insure reliability. 


Myth #10 
e military Q.P.L. list is a sound basis for picking your sup- 
pliers. 


- This is a myth because the existing Q.P.L. list means only that at 
‘one time, and perhaps on one batch which might have been handmade and 
hand picked, suppliers were able to qualify. In many cases the qualifica- 
tion requirements have changed, the qualification requirements for your 
job might be different, and the suppliers may not be able to provide the 
same reliability on subsequent batches. The Q.P.L. list is a good start- 
ing point only to pick suppliers. 


Myth #11 

Such highly contradictory specifications for reliability are coming 
out of the military services that if you comply with one you can't comply 
with any of the others. 


- If this situation exists in your operation then your original 
method of compliance was probably improper. In most cases a program for 
Seeeeerses operation can be implemented to meet any of the new specifica- 

ons. 


Myth #12 
Until someone really defines reliability, there is no point in my 
company taking it seriously. 


- The answer to this myth can be illustrated by a cartoon showing 
the living room of an exclusive men's club for retired millionaires. 
Seated about in a circle of over-stuffed chairs are a group of bored 
discussants. But contrary to the usual quiet and sedate background, every 
door and window is pouring in thick angry woodsmoke. Instead of scrambl- 
ing to find some exit throught the obviously surrounding fire, the dis- 
cussion leader quietly asks one of the other reclining members: "And 
George, how would you define smoke?" 


The moral of this is that if you are prone to wait for a good defi- 
nition, it may already be too late to escape the fire. 


Myth #13 
As in the past, free enterprise in connection with the law of supply 


and demand will provide whatever is required for us to successfully meet 
the reliability demands of the future. 


- The myth in this statement probably hinges around the word "us". 
Free enterprise and the law of supply and demand will undoubtably provide 
an answer, but not through and for those who wait to feel the pinch of 
economic dire necessity. Too little and too late is apt to describe their 
situation. Those who take the cue and lead the pack will get the juicy 
worms. 


Myth #14 
ecent past history has shown that the U.S.A. is well ahead of the 
rest of the world in the field of reliability control. 


- The personal experience of the author in Europe on repeated speak- 
ing tours and the experience of many experts traveling to Japan and the 
Far East indicate that the U.S.A. is not far ahead. In fact, at the rate 
of progress int hese foreign lands we can be left far behind in the near 
future unless we look to our laurels. I believe this is true not only 
in regard to reliability, but to production capacity and price. Our 
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vaunted lead in production capability is being severely challenged. 


Myth #15 

Industry has always been faced with the need for reliable dependable 
products. Management as always must pass this responsibility on to its 
technical organization and not get inw lved. 


- This myth hinges about the fact that top management is involved 
in reliability whether the individuals realize it or not. Every manage- 
ment decision has implications for the ultimate reliability of the pro- 
duct. The effectiveness of the operation, the efficiency of the organi- 
zation, the emphasis of the programs, the spirit of compliance to speci- 
fications, and the morale of the men all follow the example set at the 
top. Management has a big hand in reliability and must make every deci- 
sion with an awareness of its impact on reliability. 


Myth #16 
Some products are just naturally better than others, and how to 
specifically control reliability is still mostly a mystery. 


- It is true that some products are better than others for inherent 
reasons. Usually factors such as simplicity of function and type of ap- 
plication will explain the differences. But there is very little mystery 
left in reliability. Lack of information exists still in a few areas, 
but methods of control are thoroughly worked out. 


Myth #17 
Reliability is just another quality characteristic that has to be 
controlled; therefore, quality control is basically responsible. 


- A loose usage of the word "quality” makes it synonymous with the 
word "characteristic". Any quality of a product is thought to be any 
characteristic that has to be controlied. Since reliability is a product 
characteristic that has to be controlled, then, ergo, this loose thinking 
makes reliability a quality characteristic. Usually, the people that use 
this type of logic also conclude that since their quality control organi- 
zation was established to control quality, then it must obviously be re- 
sponsible for reliability. 


There are some officials in the ASQC who visualize the entire field 
of quality control in this generalized way. However, according to an 
analysis of the field in 1959 by Professor Ellis Ott of Rutgers University 
more than 85% of the profession are engaged in the historic field of fac- 
tory process and machine control. In the electronic industry this per- 
centage is probably much higher than 90%. In other words, contrary to 
what some wishful thinking would have it be, the profession of quality 
control is actually one of controlling the uniformity of a factory output 
to meet certain performance criteria as specified by engineering. Thus, 
reliability is only a quality characteristic if it is so specified and to 
the extent specified on the drawings. And the best quality control effort 
cannot achieve any higher reliability in a product than is inherent in the 
design on the drawings. 


It has been proven many times that a pee quality control operation 
can result in the production of a very uniform high quality item that is 
very unreliable. The product can be uniform and wmmply fully with the 
specifications, but be very unreliable in use. Highly purified medicine 
compounds might be given as a perfect non-electronic example of end use 
affecting reliability completely out of relation to the quality of the 
product. The quality is important, but it is only one factor in relia- 
bility. 

Many other examples in all industries can be given to show that relia- 


bility is not just another quality characteristic unless this term is used 
in an unrealistic loose fashion. 


Myth #18 
Total quality control is the answer to achieving reliability. 


- This concept is usually accepted by those who are thinking in the 
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loose terms of reliability being a quality characteristic to be controlied 
but whose experience tells them that the normal functioning area of qual- 
ity control is not broad enough to do the job. Not wishing to drop the 
word "quality" for the better term "characteristic" they add the word 
"total" to indicate that they mean something greater and quite different 
from quality control. They are thinking in terms of an over-all manage- 
ment control program which extends into ali other departments of a company 
in addition to production. They will clarify this concept by the explana 
tion that they are referring to a cradle to grave control operation. This 
operation starts up in earliest planning and has implications and inputs 
throughout every phase of development and manufacture and affecting every 
department including marketing, engineering, and through into field ser- 
vice. 


But there are some people who point out that the founding fathers of 
ASQC thought of their profession in this broad cradle to grave perspective 
and, therefore, in this broader artificial sense the added word "total" is 
unnecessary. By this argument the need is to convert the entire quality 
control profession to what the proponents of the concept mean by total 
quality control. The question then remains as to what you would call the 
present occupation and profession that guarantees a chemical purity to 
aspirin tablets. Presumably this could be accomplished by an upgraded 
inspection function and the establishment of a new profession entitled 
"Inspection Engineering". This new profession would take over the jobs 
now performed by quality engineering. Process control criteria, inspec- 
tion method planning and the analysis of factory failures, plus other fac- 
tory oriented quality engineering functions would be shifted to this newly 
titled specialty. Quality control engineering then would shift over to 
perform the functions now provided by such specialties as management engi- 
neering, operations analysis, and program and budget analysis, etc. The 
advantages of all this shifting of titles and duties to preserve the orig- 
inal intent of quality control seem doubtful. A more logical approach 
would seem to be the recognition of the important contribution that qual- 
ity control makes to process control and product uniformity and not try 
convert the established profession to something it is not. In fact, most 
existing quality control operations could benefit greatly by professional 
improvement in the historic areas of process and factory control without 
expanding into other fields of activity. 


But is total quality control in the new sense really the answer to 
achieving reliability? Agreed that the historic field of quality control 
is in the factory; it can be readily shown that this function is affected 
by design decisions, contractual aspects of marketing and even pre-pro ject 
customer planning. Thus, it can be granted that good quality control in 
its established factory control sense is always a cradle to the grave 
function. For example, a specific design might be much easier to produce, 
and, thus, result in a more uniform product than another functionally com- 
parable circuit. Again, if the process capabilities are considered during 
the product design stage, the job of quality control can be made much eas- 
ier. And a third example, unless provisions are made in the contract to 
perform the various quality control functions these cannot be implemented 
when the project reaches the production stage. So there are cradle to 
grave or total quality considerations which are important to the success 
of the usual type of quality control. But none of these factors necessar- 
ily control reliability or can assure that a product will meet a specified 
reliability goal. Other proof can be given that the total quality control 
concept is not the answer to reliability. 


Without going into this in detail here, it should be emphasized that 
reliability, maintainability, value and other product characteristic con- 
trol programs are also all cradle to grave control specialties. They all 
have implications and inputs at every project phase and to every depart- 
ment of each company. Their major differences are in type of execution 
related to the development stage of the project in which their major con- 
trol is exercised, and to the type of manpower and specialist training 
needed to perform the work. [In the majority of cases the established 
quality control people do not have either the interest, training, back- 
ground or experience to work successfully in fields other than their 
native factory process control field. To make these people responsible 
for reliability and other related product characteristics in other than 
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the factory production stage is unfair to them and is not realistic man- 
agement. 


Myth #19 
Reliability can be achieved by appointing a reliability coordinator 
under a design integration leader in engineering. 


- It is true that a great deal of reliability engineering must be 
performed at the working engineering levels. But just as quality control 
in its established factory sense has cradle to grave implications so does 
reliability control. Therefore, it is essential that reliability policy 
and top reliability control be established at an organizational level with 
influence and authority across all the departments not just in engineering. 


It has been stated that there are a dezen or more cradle to grave 
control specialties such as reliability, maintainability, ivalue, quality, 
system integration, etc. It follows that no company can afford a dozen 
Or more across the board control operations. There can be only one, and 
this must be an organization with a program to give assurance to manage- 
ment and to the customer that total quality, total reliability, total 
value and other total cortrol efforts will be exercised as needed. The 
RCA program for this is called product assurance. A booklet describing 
this is available on request. 


Myth #20 

We have appointed reliability coordinators and sent people off to 
training conferences and symposia. We are doing all we can to support 
reliability. 


- The answer to this is that I have yet to see a control operation 
that is as good as it could be. Until completely controlled operations 
become the rule rather than the exception, a great deal of work remains 
to be done. Fortunately, many of these efforts more than pay their way 
in dollars saved. 


Myth #21 

All sorts of test, measurement, and control schemes have been pro- 
posed for every phase of the company operation. We simply can't afford 
such a full reliability control program, Reliability is too expensive. 


- The cost of reliability can be discussed in four parts. Each one 
of these parts is important and has far reaching implications. All four 
will be discussed here because of their importance. They are listed on 
Slide #1. 


DOING THE JOB AT THE LOWEST COST 


A. Overall cost picture 

B. Timing of expense 

C. Project cost accounting 
D. Value engineering 


- Slide #1 - 


A. Overall Cost Picture -- The Slide #2 shows a curve plotting the rela- 
tionship between initial cost and level of reliability. Obviously, if we 
are building an equipment with low reliability we can produce this at a 
fairly low cost; however, if we are trying to build a very high relia- 
bility equipment, then the initial cost must be quite high. This is, of 
course, what makes the problem so difficult when somebody comes to you and 
says: "How much will it cost to complete a reliability program on this 
project?". You have to come back with the question "How much reliability 
are you talking about?". Obviously, if you are talking about a relia- 
bility which is going to raise the state of the art, then the particular 
cost of that particular equipment has to be very, very high initially. 
But the compromise to this is shown in the next Slide #3. 


Slide #3 shows the other side of the picture - the support cost. 
The support cost is hound to be very high if you are selling equipment 


‘ 


1961 ASQC CONVENTION TRANSACTIONS 


HOIH 
WILINI 


3HL 


- SLIDE #2 - 


290 
J 


HOIH MOT 


1S09 


- SLIDE #3 - 


PHILADELPHIA, PENNSYLVANIA 


LSOD 3HL 


291 

Sb 
a2 3 
a9 

5° 

| 

Be 
aif 


292 1961 ASQC CONVENTION TRANSACTIONS 


with very low reliability, and it will drop off and approach zero as the 
reliability goes up. Obviously, if you had 100 per cent reliability and 
the equipment never failed, you would incur no support cost. 


Thus, we have these two curves applying to each project which can be 
combined to show the over-all cost picture as in the next Slide #4, 


Note there is a certain optimum point of compromise. This is an 
idealized curve, of course, but there is an optimized point between ini- 
tial cost and support cost which must be reached as the ideal cost objec- 
tive of the reliability program. But, unfortunately, for the program man- 
ager the support cost and initial cost frequently do not come out of the 
same customer budget. This introduces the need for effective customer 
liaison and good marketing relations. Another problem for the program 
manager is that although it is very easy to point out that these curves 
do apply to projects in general, it is usually rather difficult in any 
particular project to show just where you stand on any one of these curves, 
This is one reason why a formal program for reliability data processing 
is absolutely essential. 


B. Timing of Expense -- This is certainly a very critical point as is il- 
lustrated in the next few slides. 


The tendency in the past, and very often even in the present, is to 
say: "Well, we will not bother with reliability until we really have to. 
We are going to keep our costs to the minimum."; and people say: "We cam 
not afford to do anything special, but we are going to depend on our good 
engineering techniques, our good production practices, and our good qual- 
ity control operation to make our product reliable." The result is that 
the cumulative cost can be illustrated by a curve such as is on Slide #5. 


This cumulative cost curve goes up rapidly towards the end when the 
product has been rejected by the customer as inadequate. The cost defi- 
nitely goes up well beyond the potential project billing and bemmes a 
very expensive venture including perhaps prestige loss to the company. 
This approach many times results in returning the product to the factory, 
redesigning, re-working, and rebuilding it. This is a very expensive 
operation, so the timing is bad. The effort then must be to bring the 
timing of expense further ahead on the project. 


In Siide #6 we have illustrated a situation where people say "We 
have a very thorough and capable quality control operation. We are now 
going to make our quality control a reliability control operation and put 
them in charge of making our product reliable." This is one approach that 
many people mistakenly take. It represents bad timing and is an expensive 
approach for the very simple reason that, in general, quality control op- 
erations do not control the product until a design has been released to 
production. This, of course, is too late in the game, and you can get a 
very similar cumulative cost curve which caneasily extend beyond the po- 
tential project billing. 


There are things that can be done in production to make a better 
product, but if the design is bad, quality control cannot inspect-in or 
test-in the desired reliability. If you achieve an end product and then 
have to go back to redesigning the circuits, you have a lot of hard re- 
work and it is a very expensive operation. 


It should also be emphasized that no matter how good the design in- 
herent reliability may be, the product reliability after production will 
be low unless you do have good quality control. 


These various curves that I have been showing represent common levels 
of advance in industrial experience. There is this definite trend in each 
company program development as the people learn from their own experience. 
It is a human tendency not to be able to learn from others experience, but 
to repeat one anothers mistakes. 


The next Slide #7 illustrates the cumulative cost picture for the 
next step in the trend toward better product control. 
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A reliability program is organized to start its activity during late 
design. The reason for not starting the program sooner is very often be- 
cause of the engineering tendency to say: "Do not bother me now about 
reliability. I do not even know what the circuit is going to be yet." 
The engineers themselves have a tendency to delay the controls this way 
in spite of efforts by management and other people to push them ahead. 
You can sometimes arrive at a fairly acceptable equipment this way, but 
it usually requires some patch-up and redesign. Also, the product is 
usually not so good as the product which is designed properly from the 
start in the optimum fashion shown in the next Slide #8. 


Here in Slide #8 we have essentially an optimum cost timing curve 
where you have proper expenses right from the earliest planning of the 
project all the way through and resulting in the best possible equipment 
at the lowest possible cost. There are many things that can be said here. 
For one thing, the curve is not necessarily a flat lineal curve; it might is, 
bulge near the start. Actually, the expense early in the game can very ’ 
often lower the final total cumulative cost. The.reason for this is that ; 
prevention is usually much less expensive than cleaning up, or correction 
later on. It eliminates that feedback of defective hardware; it elimi- 
nates redesign and all the other rework aspects which mean expense. If the 
designers are given the reliability responsibility, the control will come 
after the initial design. Auxiliary services and standards are needed to 
aid the designers. But, in general, to accomplish this calls for formal 
organization and we are back to program for reliability control which in- 
volves a thorough organization for reliability control in all aspects of 
the operation. 


C. Project Cost Accounting -- The third aspect of cost I want to cover 
deals with cost accounting procedures. In order to accomplish the optimum 
expense at the right time, it is essential that the program manager be 
able to put a finger on things which are not normally accountable in cost 
procedure methods. The costs that deal with preventive efforts versus 
fire-fighting and rework for example. These two costs are seldom accounted 
for separately. The usual cost accounting procedure is established for 
the simple process of billing to the customer all costs that accrue on 

his project, and essentially no effort is made to differentiate between 
preventive expense and corrective expense. This must be done if it is 
going to be possible to achieve an optimum compromise on timed expense. 


The next Slide #9 shows a bar graph of the type needed to illustrate 
relative costs for control purposes. 


Actually, just any single illustration of this nature as shown in 
Slide #9 set up showing relative costs will not tell the story either. 
You have to be able to plot this essentially in the time domain showing 
how the preventive and clean up costs vary from time to time and in accord- 
ance with your control efforts. This is a new responsibility of the pro- 
gram manager and is a problem that requires considerable extra effort on 
every project plus good data control. 


D. Value Engineering -- The fourth subject under "doing the job at the 
lowest cost” is value engineering. This is a program for a second look 
at every design to make sure that full value is received from every func- 
tion and device employed in the product. The premise behind value engi- 
neering is that value is relative to the usefulness or function served. 
In no case is value inherent in anything you buy. Value analysis, the 
tool of value engineering, is the process of comparing relative values. 


The problem to the program manager is to achieve more than apparent 
value. Many alternative methods for accomplishing a function may give 
essentially the same value until long range reliability support cost is 
considered, And it is usually very difficult to measure high reliability 
in a simple enough way to make sure that a supposedly higher value is not 
causing lower reliability. The present state of the art in reliability 
requires that value engineering be an integral part of reliability con- 
trol. 


The five major aspects of value engineering important to the program 
manager are listed on Slide #10. Here are shown the usual four functions 
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of value engineering plus a fifth which is very important to the program 
manager, i.e., "expediting the application of results from value analysis.” 


LOWEST COST THROUGH VALUE ENGINEERING 


1. Getting the facts & item costs 
2. Determining the exact function 
3. Thinking creatively 

4. Evaluating by comparison 

5. Expediting application 


- Slide #10 - 


CONCLUSION 
The twenty-one myths in reliability just discussed are only some of 
the more common ones. There are many others that might be equally impor- 
tant. 


In only a few cases has it been possible in the time allowed to do 
more than describe the myth. Awareness of what is fact and what is fic- 
tion in the new field of reliability is the big job of nearly everyone in 
industry and the military today. 


I hope this discussion has helped clarify many concepts in your minds 
and will stimulate you to further effort in finding answers to the imple- 
mentation problems in your areas of work. 


| 
| 
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ELECTRONIC RELIABILITY - A MECHANICAL PROBLEM 


Howard I. Dwyer, Jr., Manager, Quality Control 
The Bendix Corporation, Cincinnati Division, Cincinnati, Ohio 


INTRODUCTION 


Many stories in folklore deal with the wide variations between 
different interpretations of the same phenomena. Many of these stories 
point out the moral that no single interpretation is the only possible 
one. Sometimes in electronics work we fail to remember this moral. The 
current reliability problem may easily be one result of this failure. 


When an electronically-criented person looks at complex equipment 
such as a computor, a guidance system, or a radar set, he is likely to 
visualize it first as a system of circuit elements and signals. In the 
next step, he will probably consider the nature of these elements and 
their electrical characteristics, The mechanical design engineer will 
picture a collection of cabinets, racks and chassis and consider how to 
hold them in place, The man on the street, of course, sees the same de- 
vice as a weird collection of flashing lights, jittery meters, switches, 
and - sometimes - long blue sparks. All of these viewpoints are partial- 
ly correct; each is limited in some fashion. 


This paper will present a mechanically-oriented look at electronic 
equipment. The viewpoint, although mechanical, is not limited to the 
characteristics generally described as mechanical, A piece of equipment 
will be considered as a complex structure serving a structural purpose. 
An approach similar to that which might be used in discussing a landing 
gear, an airframe, a bridge or a truck's steering mechanism will be 
used. The desired result is an appreciation for, and an awakening of 
interest in, the possible contributions of a structural approach to 
equipment reliability. 


Remembering the moral mentioned earlier, we shall refrain from say- 
ing that structural integrity is the only criterion for equipment reli- 
ability. Electrical characteristics of systems, circuits and components 
must be engineered to promote reliable operations. Reliable operation 
is impossible, however, when elements of systems, circuits or components 
fail structurally. Our basic premise will be the somewhat heretical 
statement that most, if not all, electrical equipment failures are the 
result of mechanical breakdown. 


SYSTEMS 


A large electronic equipment is a complex system of electrical parts 
and circuits. It is also much more than that. When you adopt a mechan- 
ical viewpoint you see a complicated non-homogeneous structure rivaling 
the largest mechanical structure in its complexity. In a typical system 
you find soldered and welded joints, roller joints, pin and rivet joints, 
friction joints and bolted joints. There are cantilever, simple and 
continuous beams arranged singly and in odd shaped trusses. A variety 
of tension, compression, shear, torsion and bending loads are applied 
impulsively, steadily, dynamically and repetitively. Interacting ele- 
ments are coupled statically and dynamically. Although few parts are 
designed to be moving parts, many sections are effectively moving parts 
with small excursion because of deflections and deformations. Under the 
right conditions any part of this system can fail as a mechanical struct- 
ure. When this happens, the system will usually fail as an electrical 
structure as well. The next few paragraphs will explore some conditions 
which promote electrical failure as a secondary effect of a primary 
mechanical failure. 
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HUMAN FACTORS 


The human element enters the system at the first fabrication stage. 
The best assemblers in the world will occasionally forget how to do some 
part of a job or they will misinterpret their instructions. Operations 
will be missed or done incorrectly under the best conditions; if condi- 
tions are not the best, errors will be even more common. Every quality 
control investigator has had the experience of looking at a defective 
piece with the feeling that the assembler must have gone completely 
insane to make it that way. Unfortunately every assembly error is not 
in this class. All too frequently we find a design or a process sheet 
which seems to require a combination of Superman and a magician to figure 
it out, let alone put together. A previous paper (1) listed some points 
which should be considered to avoid this unpleasant situation. By their 
very nature these points may be skipped in the design stage. They are 
the mechanical design details which change a design from something to 
fight in production to something which is a joy to build. The main rea- 
son they are overlooked is that they are obvious - after they are men- 
tioned. For example, everybody knows that you must be able to see a 
terminal if you are going to do a neat soldering job on it, that pushing 
a wire aside to reach a mounting screw is potentially a source of broken 
wires, and that reversed cables usually cause trouble. Designs are still 
made every day with these faults and many others designed into the system, 


System maintainability and operability have received considerable 
attention in the design stage during the past few years. The effects of 
both on system reliability in service have been explored extensively, and 
checklists of good practice have been developed. The areas of assembly 
and handling have not received the same degree of specialized attention, 


Correlation between an impulsive load applied when a chassis is 
dropped on the loading dock, or even when an access door is slammed shut, 
and electrical failure may be difficult. Correlation between the same 
load and a fractured component is relatively easy. In between these two 
levels is the situation where the impulsive load shifts a component 
slightly in its mounting thus putting extra stress on the component leads 
Snapping a diode into a clip, bending a lead at the surface of a glass 
header or warping the mounting flange of a relay slightly during instal- 
lation may not cause visible damage or obvious malfunction. Internal 
stresses and deformations are set up which eventually will lower the: 
ability of the part to withstand additional loads. 


Although we all pride ourselves on the ability and skill of our 
production people, perhaps the safest course during the design stage is 
to assume that they are all illiterate dunderheads waiting for the chance 
to do something the wrong way. Our designs and processes should, when- 
ever practical, preclude a failure due to sloppy workmanship. 


ENVIRONMENTS 


A system must operate and survive in a series of environments. 
These environments consist of all natural and artificial phenomena in 
the system's immediate vicinity. In addition to standard environments 
such as vibration, shock, atmospheric pressure and temperature, moisture 
and corrosive atmospheres, we might consider time and gravitational 
attraction as well. Space systems programs have become aware of the 
effects of low gravity, but most of us still take gravity for granted, 
Gravitational attraction exerts an appreciable force on all parts of our 
systems; these forces must be reacted within the system, and hence 
stresses are introduced. Time, or duration, is an important environment 
in that chemical processes and cumulative effects are affected. No real 
substance is completely inert with respect to duration. Changes take 
place in systems which can only be explained and studied by considering 
duration as an environment. 


The engineer must realize from the start that he can not eliminate 
the environment. The best he can hope to do is protect the system from 
harmful environmental effects. Because most environmental effects act 
through mechanical means, protection is almost invariably mechanical. 
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Shock mounts, vibration isolators, thermal barriers and adequate struc- 
tural strength are all means to prevent overstressing electrical parts. 


STRUCTURE 


A system consists of two basic types of members. The active ele- 
ments are electrical components while the passive elements are primary 
structure such as racks, cabinets, chassis and other framework. It is 
characteristic of much of our modern electronics equipment that a thou- 
sand dollars worth of electrical system is wrapped up in ten dollars 
worth of sheet metal. A program which aliows two years to develop a 
system will spend two weeks in developing primary structure. In some 
cases more attention is given to markings on the front panel than is 
given to the strength, rigidity and durability of main equipment support. 


Primary structure must do more than merely hold the system together 
The primary structure transmits all external environmental loads, and 
provides most of the protection against these loads. Improper design can 
provide transmissibilities and force magnifications orders of magnitude 
greater than those found in a good design. Energy received from both 
external and internal environments must be dissipated. If this is not 
accomplished in the structure, it must be done within the active ele- 
ments. Structural design must be a series of compromises between ade- 
quate protection and excessive weight and cost. A cartoon widely circu- 
lated in aircraft design sections shows the “ideal" airplane as envision- 
ed by each design section. In this cartoon, the structures section's 
airplane consists of massive I-beams riveted together with gusset plates 
and braced with chain. Such an approach is as unrealistic for electronics 
as it is for airframes. The opposite extreme with thin members, flimsy 
hinges, and large flexible panels is equally unrealistic. 


COMPONENTS 


Many system failures aie caused by failure of some critical elec- 
trical component. Tubes become noisy, resistors change value, diodes 
epen up, relays stick, and transformers develop shorted windings. These 
failures, and a host of others, are mechanical failures rather than elec- 
trical. In fact, with the exception of some types of semiconductor 
failures, all component failures are mechanical in nature although the 
end result is an electrical failure. 


The logic behind this statement becomes evident when the internal 
structure of electrical components is made the area of interest instead 
of the purpose of the component. For example, the purpose of a tube is 
essentially either rectification, amplification or controlling a signal. 
How well the tube serves its purpose is dependent on the mechanical 
characteristics of envelope integrity, element continuity, element spac- 
ing, cathode coating uniformity, and foreign particle content. Assuming 
that the correct tube type was chosen for the circuit application, cir- 
cuit performance is dependent only on these mechanical features. The 
standard descriptions of tube failure, e.g. low gain, noisy, intermittent 
operation, are really descriptions of symptoms. The actual failure of 
the tube is the failure of the tube structure. True tube failures would 
include element displacement, broken element or lead, abraded spacer, 
defective seal, and flaked cathode coating. 


This discussion would be academic - even pedantic - were it not for 
the insight such an approach gives to the relationship between applied 
loads and component failures. Loads applied to the component structure 
generate mechanical stresses in the structure. If these.stresses exceed 
the strength of the structure, the component fails. The means by which 
the loads are applied may be electrical, e.g. current surge, thermal 
e.g. high ambient temperature, or mechanical, e.g. shock, without obscur- 
ing the relationship. 


Since the objective of reliability programs shonld be the reduction 
or elimination of equipment failures, this relationship should be ex- 
plored more fully. 
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COMPARTMENTED DESIGN 


In the introductory remarks reference was made to some of the 
different viewpoints possible in visualizing an electrical system. There 
has been a movement in recent years to capitalize on this by processing 
a new design through various stages of design review and redesign by 
different segments of the organization. New designs are scrutinized, 
modified or approved by disinterested designers, reliability engineers, 
value analysis committees, field service engineers and a host of others. 
Almost every conceivable combination and permutation has been tried at 
least once. When properly done, this process can assist in developing 
a device that actually works in service as well as it did on paper. A 
recent paper (2) discussed design reviews in general and some effective 
types of review programs. As long as design review does not result in a 
"compartmented" effort, the idea is sound. If design reviews or similar 
programs develop a series of independent designs tacked together rather 
than a single integrated design, they will defeat their own purpose. 


In a hypothetical, but all too possible, program we might find a 
design passing successively through complete redesigns in which each 
section has followed its own particular bias without respect for previous 
work, By the time the circuit designer, applications engineer, mechan- 
ical features engineer, packaging engineer, reliability engineer, main- 
tainability engineer, producibility engineer, value analysis section, 
cost reduction committee, field service technician, inspection processing 
engineer, weight control analyst, tooling engineer, and all the rest have 
each taken a stab at “improving the product", the end item will look like 
a patchwork quilt. Any resemblence to the original design is purely 
coincidental, 


This paper does not advocate a compartmented approach. Consider- 
ation of mechanical characteristics must be as much a part of the de- 
signers' original work as determination of voltage levels and waveshapes, 
An electronic system designer has as little right to sav that fatigue 
life of a tube spacer is not his concern as a hydraulic system designer 
has to disregard the bursting stress of a piece of tubing. Should the 
objection be raised that specialization is necessary in modern industry, 
the only answer is that a designer must not be so specialized that he 
can not design properly. We do not expect a system designer to be a 
specialist in inductance and be completely ignorant of capacitance. Why 
then should we accept the idea that this designer may legitimately claim 
to be a specialist in electrical characteristics and ignorant of bending 
stresses? 


Consideration of mechanical aspects of electrical designs must be 
as basic to system and circuit designers as electrical aspects. Quality 
control, reliability, manufacturing, processing and other sections must 
expand their fields of interest beyond those required for purely elec- 
trical considerations. None shonld fall into the trap of treating each 
consideration purely as a separate entity. Any review or study of the 
mechanical side of a design should be integrated with the electrical 
side. This applies equally to study of production problems, service 
failures and original designs. 


RELIABILITY DEFINITION AND PREDICTION 


Reliability is generally defined, at least in Reliability circles, 
as "the probability that the device will perform its intended function". 
The phrases "for the intended period" and "when used in the intended 
manner" are usually added, but these are tautological. Using this defi- 
nition, reliability requirements have been specified, predictions have 
been made, and equipment has been redesigned or scrapped. The definition 
is useful in establishing a figure of merit for a device, but is not 
very helpful in designing or building anything. 


For discussion purposes, a new definition for "reliability" is 
proposed: 


"The reliability of a device is the probability that applied 
loads will not cause stresses in an element of the device to 
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exceed the failures strength of the element." ; 
4 

This definition is based on four fundamental ideas: 


(1) An element or a device has failed when it ceases to perform 
the function for which it was intended, 


(2) A device fails if, and only if, some element of the device 
fails. 


(3) An element fails if, and only if, a particular stress or set f 
of stresses exceeds the strength of that element. ee 


(4) A stress is induced in an element if, and only if, a load is 
applied to the element. ’ 


Under these conditions the probability of no failure is obtained 
automatically from consideration of applied loads and element properties. 
Since, as we have seen, time is simply another type of load application, 
the time factor in the "classical" reliability definition is included, 


If we use this proposed definition, we have a means of studying 
and improving the reliability of systems and components. The definition 
focusses attention on basic questions regarding failure: 


(1) What are the expected loads? 
(2) What stresses will these loads generate? 
(3) What is the strength of the piece? 


Testing and analysis of systems, environments and properties can 
provide, within acceptable engineering accuracy, these answers. Natur- 
ally both loads and strengths will have statistical distributions; cal- 
culation of probabilities under these conditions is routine, 


Reliability analysis then becomes a question of straight engineer- 
ing similar to that used in the design of reliable systems such as 
bridges, skyscrapers, airframes and steam engines. Fach element in a 
system can be investigated, either analvtically or by experiment, to 
determine what loads it must carry, how the loads will be applied, and 
what will be the results of applying these loads. For example, let us 
consider a resistor used in an application where resistance must not 
vary appreciably. To compute the reliability of the resistor, we might 
follow these steps: 


(1) Question: Why does resistance vary? 
Answer : Properties change with heat. 


(2) Question: What heats a resistor? 
Answer : (a) Heat from surrounding environment, 
(b) Internal heat generated by passage of 
current through a resistance. 


(3) Question: What current variation can we expect? 
Answer : Derive from circuit analysis and measurement, 


(4) Question: How does the resistance value vary with heat? 
Answer : Plot ohmeter readings vs. thermocouple read- 
ings. 
(5) Question: What is a failure? 
Answer +: Resistance value outside specified limits, 
Therefore heat outside computed limit. 
Therefore current outside computed limit. 


(6) Question: What is the reliability of this resistor in 
this application? 
Answer : Compute from results of steps (3) and (5). 


Since this was an example intended only to outline an approach, it 
was somewhat over-simplified. Other factors, such as variation of pro- 
perties with time and non-homogeneity of resistors could be included if 
desired, 
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The classical definition of reliability can be used to set goals 
and measure accomplishments. It can not be used to predict what you will 
achieve for new designs or to tell you how to take corrective steps. The 
proposed definition and study of mechanical factors overcome both of 
these objections. 


CONCLUDING REMARKS 


The ideas presented in this paper are not necessarily new. In fact 
most of them are as old as engineering itself. In a dynamic culture, old 
ideas are sometimes overshadowed by new ones or they are forgotten in our 
enthusiasm for moving ahead. The idea that an electrical part is a mech- 
anically stressed member as well as a functional circuit element may be 
new to some and old to others. If this paper provides the impetus for 
designers, quality engineers and reliability specialists to consider 
failures from a different perspective it will have served its purpose. 
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OPTIMIZATION OF EQUIPMENT EFFECTIVENESS 


Anthony Drummond 
ARINC Research Corporation 


In the many fields of responsible manufacturing, an endeavor to pro- 
vide "Product Value" may be recognized as a common pursuit. The precise 
image of this ideal and the methods used to attain it will vary with the 
product, and will become more difficult to define as the complexity of 
the product increases -- clearly, value criteria inay be determined and 
value goals achieved more easily in the manufacture of garbage cans than 
of airliners. At the summit of such an hierarchy, the problems of 
trading-off performance with cost become acute when military usage intro- 
duces possible cataclysmic consequences of failure or error. 


Under Air Force auspices, ARINC Research Corporation has been en- 
gaged in a program directed towards developing techniques for gaining 
maximum product value in the specific area of airborne electronic equip- 
ment. This is an extensive project, the scope of which is illustrated to 
some degree by Figure 1, which shows ARINC's concepts of the factors 
which must be considered in a study of equipment value. 


Equipment 
Value 
Requirements 
Effectiveness on the Broader Dougpegnent Cost 
System 
FIGURE 1 


FACTORS AFFECTING EQUIPMENT VALUE 
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"Effectiveness" is a measure of the equipment's ability to fulfill 
the requirements of the job for which it is purchased. The second 
factor -- "Requirements on the Broader System" -~ encompasses equipment 
characteristics such as weight, volume, power requirements, which tax the 
supporting environment. "Development Time" is included since, in the 
consumer products field, its careful control may well enhance the value 
of a product by allowing its marketing ahead of the product of a competi- 
tors; a. short Development Time for a military equipment may be the essence 
of its value to the defense structure of the Nation. The "Cost" factor 
included in this breakdown may be considered purely in terms of the dol- 
lars and cents expended in initial costs and subsequent upkeep, although 
it is conceivable that, for military equipments, it may sometime become 
necessary to extend the concept of cost to include such other items, per- 
haps, as a raw material in short supply or human life itself. All these 
factors must be weighed and judicious compromise applied in order to 
achieve a design leading to the best equipment value for the circum- 
stances. 


The provision of methods for predicting the trade-off relationships 
which will facilitate such compromising is a long-term aim of ARINC's 
project. In this pursuit, techniques for quantifying, measuring, and 
fully understanding the elements of the four factors affecting equipment 
value must be developed. Information obtained by the application of such 
techniques to existing equipments must then be used in turn to develop 
methods of synthesizing values for these factors for new equipments while 
they are still in the conceptual and early-design stage. Several such 
estimates or predictions computed for each factor with different condi- 
tional values assigned to the other factors, will provide the trade-off 
functions upon which the early policy and design decisions may be based. 


Of course, when working towards an ultimate goal of equipment value 
maximization, it does not suffice to provide only a capability of pre- 
dicting the contribution of each factor. Attention must be given also to 
developing methods of optimizing the contributions themselves. In the 
area of Effectiveness, where most of the work to date has been concen- 
trated, studies directed towards attaining these two ends are very 
closely interwoven and have proceeded concurrently. Figure 2 is a sim- 
plification of the framework within which the study of Effectiveness, 
from both viewpoints, is progressing at ARINC. 


The breakdown shown provides two channels of research -- Design 
Adequacy and Mission Reliability-Operational Readiness. Without going 
into precise definitions, the former concerns the equipment's ability to 
do its job if working correctly; the other, the probability that it will 
work correctly when required and the probability that it will work for a 
given time. In Figure 2, the detailed analysis afforded the latter chan- 
nel, while Design Adequacy remains singularly uncomplicated, suggests 
that research effort in the realm of quantification and prediction tech- 
nigues has not been equally divided between the two. This is true, and 
this circumstance, rather than any wish to belittle the importance of 
Design Adequacy, explains the orientation of the remainder of this paper 
toward our studies of the Mission Reliability-Operational Readiness as- 
pect of Effectiveness and, specifically, its essential components: reli- 
ability and maintainability. 


At this level, the tactical approach to ultimate value maximization 
has been the development of techniques for reliability and maintainabil- 
ity prediction. Such techniques not only will aid in providing the 
initial predictions mentioned previously, but also will play an important 
role in the optimizing of reliability and maintainability (and thereby, 
ettectivennes? during the design, prototype and testing phases of an 
equipment's development. During this time, periodic predictions of field 
reliability and maintainability, based upon the constantly changing data 
from drawings, part and unit tests, system tests, etc., will disclose 
with what proximity the desired field levels for these characteristics 
are being approached, and in what areas design improvement effort should 
be concentrated. Although the future tense has been used here, it will, 
of course, be realized that the use of reliability prediction techniques 
developed to date is already widely recognized, particularly for the 
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Design Mission Operational 
Adequacy Reliability Readiness 
| 
Storage Free 
Availability Time Time 
Reliability 
Maintainability 
| 
| | 
Active = 
gistic Administrative 
— Time Time 
FIGURE 2 


CONCEPTS ASSOCIATED WITH EQUIPMENT EFFECTIVENESS 


design-improvement purpose mentioned above. The time is at hand, more- 
over, when documented prediction of reliability is being specified as part 
of military acceptance procedures, bringing closer the day when such 
assurances will be required at the over-all Value level. In this connec- 
tion, it must be realized that as equipments become more complex and 
potential use environments become more difficult to simulate, ultimate 
performance and value become increasingly difficult to estimate without 
some such detailed prediction method. 


The foregoing discussion having placed the roles of reliability and 
maintainability prediction studies in proper perspective within the long- 
term plans for Value maximization, we will proceed to a detailed discus- 
sion of these studies. The mention of reliability prediction's current 
use and anticipated further use illustrates that studies in this area are 
well ahead of those in the maintainability field. Reliability prediction 
in practice is well established -- even if not im all cases well under- 
stood -- while maintainability prediction is still a curiosity. For this 
reason, rather than reiterate in extenso the details of reliability pre- 
diction methods, we will merely recall the basic approach and a particu- 
lar ramification thereof, and provide a fuller treatment of our work to 
date on maintainability prediction. 


In essence, reliability prediction during equipment design and devel- 
opment is achieved by the summation of established part failure rates 
applied to every part in the equipment. The total failure rate so 


309 
| 
& 
4 


310 1961 ASQC CONVENTION TRANSACTIONS 


obtained can be converted by use of the exponential equation to the pro- 
bability of successful operation for a given time or, more popularly, to 
an MIBF. A serious problem concerning the interpretation of these pre- 
dictions arises at their very root, due to the many types of "established" 
failure rates being used and the varied treatments they receive. ARINC's 
prediction method uses rates derived from our own extensive field surveil- 
lance of military equipment. Generally speaking, all removals to effect 

a repair are counted as failures and included in ARINC's failure-rate 
computations. At the point of final summation, correction is made for 
"cluster" removals and an inclusive rate is inserted to account for re- 
pairs made by adjustment. Final corrections are made to account for the 
complaint-repair differential. Another procedure, however, discounts all 
but catastrophic failures in deriving the part failure rates and making 
predictions therewith. In addition there are failure rates generated by 
part manufacturers and equipment manufacturers under all kinds of physi- 
cal conditions and criteria of failure. No quarrel is found with the use 
of any of these rates and procedures so long as the resultant predictions 
are not misinterpreted. For instance, a procedure which predicts only 
malfunctions due to catastrophic failures will not predict field reliabil- 
ity accurately; malfunctions due to deterioration, need for adjustment, 

or other aspects of what is labeled ineffectual maintenance, will not be 
included. This procedure will predict, however, the reliability supplied 
by the manufacturer when influences beyond his control are not considered. 
On the other hand, the failure rates and technique used by ARINC will 
build into a prediction an accounting for field influences, and by so 
doing will best predict field reliability. The point to be made is that 
a reliability prediction is of little value without information on the 
failure rates and procedure by which it was generated. 


An interesting illustration of the type of circumstance which as- 
sures discrepancy between the two types of prediction just referred to, 
also provides an excellent transition to maintainability discussion, in- 
asmuch as it establishes that maintainability not only has its direct 
influence on svailability but has a secondary influence by way of its 
effect upon reliability. During ARINC's current surveillance of an Air 
Force Bomb/Nav system, it has been observed that the majority of mal- 
functions occur during the first one tenth of a mission's flight time -- 
in fact, a large part occur immediately at take-off. It is ARINC's con- 
tention that this higher rate of malfunction early in flight is not 
caused by take-off stresses as might facilely be concluded, but rather by 
malfunctions which actually occurred during previous in-flight or ground 
operation and which were not corrected by maintenance, This explanation 
is supported by evidence supplied by two conditional probabilities de- 
rived from the observed data. 


The probabilities are defined as follows: 
(1) P (SIS,) 
The probability of successful flight (no in-flight malfunction), 


given that there was no malfunction on the preceding flight and 
therefore no corrective maintenance action between flights. 


(2) P (S|M,) 
The probability of successful flight, given that there was a 


malfunction on the preceding flight and therefore there was a 
corrective maintenance action between flights. 


On the basis of data collected over a twelve-month period, computa- 
tions of the two conditional probabilities disclosed that the MIBF under 
the first condition was:34% higher than that under the second condition, 
Clearly, the probability of a successful flight is lower if a malfunction 
occurred on the preceding flight. The consistency of this relationship 
ae the whole twelve months represented lends authority to the 
conclusion. 
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This ever-present aspect of imperfect maintenance is accounted for 
in the prediction technique developed and used by ARINC. Such a predic- 
tion for this equipment would estimate a probability of successful flight 
within the limits of the two conditional probabilities referred to above, 
while a prediction which, by the nature of its basic inputs, was re- 
stricted to estimating the unreliability due only to catastrophic part 
failures would provide an estimate well above the more optimistic of the 
two probabilities. 


The imperfection in maintenance illustrated above is not necessarily 
the fault of maintenance personnel, but often is the result of maintain- 
ability problems beyond their control. It is difficult, for instance, to 
repair a malfunction whose symptoms cannot be observed in the ground en- 
vironment. Maintainability, then, not only concerns the time required to 
correct a malfunction, but also the probability that the malfunction can 
be observed and recognized by maintenance personnel. It is the time as- 
pect, however, which must be the subject of maintainability prediction 
techniques; consideration of the latter symptom observation must perforce 
be included in the reliability prediction. Work on the development of 
maintainability prediction techniques has proceeded at ARINC along a line 
similar to that used for reliability prediction. The maintenance activi- 
ties on a group of Air Force Bomb/Nav systems in field use have been 
under observation for approximately one year. The purpose of this sur- 
veillance is to accumulate data which will provide the basis for descrip- 
tion of the relationships between physical and circumstantial character- 
istics, and the length of time required to perform elemental activities 
germane to system maintenance in the three broad areas of 


(1) Administrative Time (inactive Down-Time) 
(2) Logistic Time 
(3) Active Repair Time 


The data will also be used to establish the probability that eagh such 
elemental activity will occur as part of a period of system "down time". 
For the system under observation, the lowest level which can be con- 
sidered is the unit, since almost no system maintenance is performed 
below that level. Similar relationships for parts must be established 
from other surveillance activities for systems which are maintained at 
the part level. The physical and circumstantial characteristics which 
seem reasonable items to be considered with reference to a unit are 
weight, volume, aspect ratio, number of visible or audible outputs and 
the like. Possibly more complex and seemingly unlikely are the charac- 
teristics which must be considered as influencing Logistic Time and ac- 
tivities of Administrative Time. The elemental activities referred to 
are a list of some 100 activities associated with Administrative Time and 
with Active Repair Time grouped into the following six categories: 


(1) preparation 

(2) symptom verification 
(3) fault location 

(4) part procurement 

(5) repair 

(6) final test. 


Logistic Time is not subdivided into elemental activities, but is treated 
as an elemental activity itself. 


The first major analysis of data will be made in the near future, 
since sufficient observations have been collected to generate many hun- 
dreds of equations of the following model: 
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where t = time required to perform the particular elemental activity 
in this observation, 


%) on = values quantifying physical or circumstantial character- 
ig istics of the unit or activity involved in this observa- 
tion, 
on ® unknown factors representing the relative influence of 
x on t 
1.<8 


a9 = unknown constant. 


The equations will be grouped by elemental activity and, assuming 
that the hypotheses underlying regression analysis are satisfied, regres- 
sion analysis will be programmed on a computer for each group to obtain 
estimates of the regression coefficients tao .n) and the standard error 
of estimate. A characteristic whose regression coefficient for a partic- 
ular elemental activity is not significantly different from zero may be 
considered to have no bearing on that activity and dismissed from the 
equation. The estimates of a9,., remaining for each elemental activity 
can then be used in the same model to predict the distribution of times 
required to perform this activity for a new system. The values for x), \y 
will be obtained from specifications or knowledge of each unit, part or 
activity of interest. 


It will have been recognized, perhaps, that a very difficult problem 
is faced in conceiving and quantifying all the pertinent items symbolized 
by x . If this endeavor meets with success, however, knowledge of all 
the physical and circumstantial characteristics to be built into or to 
provide environment for a new system will permit prediction of (1) a dis- 
tribution of times for Logistic delay, (2) a distribution of times for 
performance of each one of the elemental activities of Administrative 
Time, and (3) a distribution of times for each one of the elemental ac- 
tivities of Active Repair Time as applicable to each unit or part (or 
classified grouping of units or parts) of the system. At this point the 
problem is reduced to one of combining these basic distributions in such 
a@ manner as to provide an accurate prediction of the over-all distribu- 
tion of times required to repair the system. 


Assuming that the times required to perform the elemental activities 
are independent of each other, an adequate solution to the combinatorial 
problem apoears to be provided by employment of a Monte Carlo technique. 
Any number of discreet values representing the time required to complete 
a total maintenance action can be synthesized for the new system by this 
technique. The synthesis involves the use of the basic predicted distri- 
butions of times required to perform the elemental activities, and a dis- 
tribution-selection discipline imposed by the probability of failure for 
each unit (or part) and the probability of occurrence of each elemental 
activity (the former probability is derived from a reliability predic- 
tion; the latter, by analysis of the new system and reference to past 
experience). The distribution of times so synthesized will constitute 
the predicted distribution of total system maintenance times. Gratifying 
results have been obtained so far from exercises designed to test the 
efficacy of such application of Monte Carlo methods. 


Such is our theoretical and practical approach to developing a meth- 
od of prediciing maintainability. As studies progress, it may become 
evident that the characteristics controlling some time patterns are too 
intangible to define or quantify, in which case compromises with perfec- 
tion must be endured. On the other hand, it may be determined that other 
time requirements will remain unchanged in association with any system, 
at any time, in any place. In such cases, prediction is reduced to its 
simplest terms. 
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Reliability prediction is here; maintainability prediction is not 
far behind. With these two capabilities available, studies in the 
Mission Reliability-Operational Readiness area will approach fruition. 
Design Adequacy must then receive full attention in order to fulfill the 
aim of Effectiveness prediction and optimization. Of the four factors 
affecting Equipment Value, Effectiveness is considered to offer the 
greatest and most time-consuming problems to be solved. The other three 
factors are believed to be more receptive to treatment by adaptations of 
established methods and, hence, the road to Value prediction and maximi- 
zation will be considerably more than one quarter covered when Effective- be 
ness studies are completed. 
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COMPUTING EQUIPMENT USED FOR 
STATISTICAL STUDIES IN THE METALS INDUSTRY 


D. d. W. Allan 
American Iron and Steel Institute 
and 
F. G. Norris 
Wheeling Steel Corporation 


This paper is a report on a survey conducted by the Metals Technical Committee of 
the American Society for Quality Control to ascertain the types of computing equipment 
available to individual members of the Committee and the types of equipment which would 
be used by them in processing data for several specific statistical methods. 


BACKGROUND 
The interest of the Metals Technical Committee in the use of computing equipment 
is evidenced by the convention papers concerning the use of such equipment sponsored by 
the Committee since 1953 (references 1, 2, and 4 to 11 inclusive). 


The use of the survey procedure to develop information within the Metals Technical 
Committee was first employed in 1953 with reference to statistical methods used in the 
metals industry. The results of that survey were reported at the 1954 convention 
(reference 3). The survey results being reported in this paper were collected along 
similar lines to those employed in the 1953 survey. 


SURVEY RESULTS 
A questionnaire, example attached as Exhibit I, was sent to the 110 names which 
appeared on the Metals Technical Committee mailing list as of February 5, 1960. Replies 
were received from 76 members of the committee; these replies are the basis of this 
report. 


All of the replies received have been summarized in Exhibit II. Equipment cate- 
gory I was for "other equipment" such as X-Y plotters and analog computers. The only 
item reported which could not be described by one of these examples was the abacus. 


The most frequently received reply for each method, for each degree of urgency, 
and for each data volume has been summarized in Exhibit III. 


BIAS 
In drawing conclusions from the data, the following conditions should be kept in 
mind: 

a) The 76 replies contain differences in interpretation of the questionnaire 
so that there is no assurance that all of the replies are answers to the 
same questions. 

b) The population sampled was the membership of the Metals Technical Commit- 
tee which membership is not necessarily a representative sample of the 
quality control personnel of the metals industry or industry in general. 


SUMMARY 
The data obtained on the availability of data processing equipment indicates that 
most of the repliers had a desk calculator immediately available. About half of the 
repliers had a medium scale electronic computer available within the plant. Large 
scale equipment when available is located outside the plant at the corporation or 
service bureau level. 


The degree of availability of the various types of equipment obviously influenced 
the choice of the particular types of equipment to be used in the solution of the 
various statistical problems presented in the remainder of the questionnaire. The 
replies received, however, indicate that most problems, even those of complex nature, 
are still being solved with relatively simple equipment when adequate time is 
available. 
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EXHIBIT I 


Metals Technical Committee - American Society for Quality Control 


SURVEY OF EQUIPMENT USED TO PROCESS DATA 
IN STUDIES EMPLOYING STATISTICAL METHODS 


Questionnaire 


1. Authority. The Metals Technical Committee of the American Society for Quality 
Control authorized this survey in Pittsburgh on February 5, 1960. 


2. Purpose. 
a) To collect information on the types of equipment available to individual mem- 
bers of the Metals Technical Committee for the processing of data in their 
statistical studies. 


b) To determine the types of data processing equipment used in processing data 
for several specific statistical methods. 


c) To prepare a summary of the information collected for presentation before the 
membership of the Society at its 1961 National Convention. The identification of 
individual contributors will not be revealed in this summary. 


3- Coordinating Task Force. 


F.G. Norris - Metallurgical Engineer - Statistics 
Wheeling Steel Corporation 
Steubenville, Ohio 


D. H. W. Allan - Metallurgical Statistical Analyst 
American Iron and Steel Institute 
150 East Forty-Second Street 
New York 17, New York 


4. Participants. The participants in this survey are restricted to those members of 
the Metals Technical Committee of A.S.Q.C. who voluntarily submit data. 


5. Equipment to be Surveyed. 
a) In order to develop information regarding the data processing equipment avail- 
able to the individual members, page A of the attached questionnaire has been 
prepared. 


b) The equipment has been categorized into seven basic groups as shown below. 

These groups have been defined by reference where possible to IBM equipment for 
convenience of comparison purposes only. These references do not restrict the 

participants to that particular manufacturer's equipment. 


Category Equipment Example 
A Manual methods only Paper and pencil 
B Adding machine -- 
Cc Desk calculator Friden, Marchant, Monroe 
D Desk computer IBM 610, IBM 1620 
E Punched card sorter and tabulator -- 
F Calculating punch IBM 602, IBM 604, IBM 607 
G Medium scale electronic computer IBM 650 
H Large scale electronic computer IBM 700 series or larger 
I Other (specify) Includes X-Y plotters and 


analog computers 


In addition, a category X has been provided for the situation where the 
answer might be "would not attempt within limitations of time and avail- 
able equipment." 


ec) Information regarding the present availability of the above equipment to each 
member has been requested on page A of the questionnaire attached. The avail- 
ability has been categorized at five levels as shown below: 
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EXHIBIT I (continued) 


Category Description 
I Not presently available 
II Immediately available 
III Available on request within plant 
IV Available on request within corporation outside plant 
Vv Available on request outside the corporation on a service bureau basis 


a) Information regarding the availability of programming for the various statis- 
tical methods also has been requested on page A of the questionnaire by use of 
the following three categories: 


a. No prepared programs available 
b. A few basic programs available 
c. Most programs required available 


e) Information regarding the sources of programming for the various statistical 
methods also has been requested on page A of the questionnaire by the use of the 
following four categories: 


ad. Write ow 

e. Manufacturer 

f. Users' group 

@- Personal contacts 


Statistical Methods to be Surveyed. 

a) In order to determine the types of the data processing equipment which the 
individual members use customarily in processing data for several specific sta- 
tistical methods, pages B, C, and D of the attached questionnaire have been 
prepared. 


b) The statistical methods have been categorized into three main groups, namely: 


1. Distribution Statistics 
2. Correlation and Regression 
3- Analysis of Variance 


Each group contains a number of specifically identified statistical methods for 
which information is required. 


c) The participant is requested to indicate which of the presently available 
equipment categories is or would be used, if required, for problems of specific 
magnitudes each with definite degrees of urgency. The degrees of urgency are 
as follows: 


I Answer required in 1 day (immediately) 
II Answer required in 7 days (fairly soon) 
III Answer required in 30 days (no hurry) 


a) The principal equipment which is or would be used, if required, for each situ- 
ation on pages B, C, and D of the questionnaire is to be indicated by the use of 
the equipment category code shown in section ‘Sb above. 


Submission of Completed Questionnaire. 
a) Two copies of the questionnaire are attached (pages A to D inclusive), one 
copy for the participants' file and one for submission. 


bd) The completed questionnaire is to be forwarded to F. G. Norris, Metallurgical 
Engineer - Statistics, Wheeling Steel Corporation, Steubenville, Ohio, no later 


than April 1, 1960. 
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EXHIBIT I (continued) 


Metals Technical Committee - American Society for Quality Control 
Survey of Equipment Used to Process Data in Studies Lo Statistical Methods 


QUESTIONNAIRE - PAGE A - DATA PROCESSING EQUIPMENT AVAILABLE 


‘Submitted By: Date 


Pro, 
Availability Sources 


Equipment Availability 


CHECK 
APPROPRIATE 
BOXES 
FOR 

EACH 
EQUIPMENT 
CATEGORY 

SHOWN 

BELOW 


Equipment Category 
(Refer to Section 5b 
for Examples) 


Not available 

Immediately available 

Available on request within plant 
Available on request within corpora- 
tion outside plant 

Available on request outside corpora- 
tion on service bureau basis 

No prepared programs available 

A few basic programs available 

Most programs required available 
Users Group 

Personal contacts 


Manufacturer 


Write own 


al 
< 
o 


A |Manual Methods Only 


B | Adding Machine 


Large Scale 
Electronic Computer 


Other (Specify) 


319 
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3 
Punched Card Sorter 
and Tabulator | 2 
Medium Scale 
Electronic Computer 
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EXHIBIT I (continued) 


Metals Technical Committee - American Society for Quality Control 
Survey of Equipment Used to Process Data in Studies Employing Statistical Methods 


QUESTIONNAIRE - PAGE B - DISTRIBUTION STATISTICS 
INDICATE PRINCIPAL EQUIPMENT USED IN EACH BOX USING THE FOLLOWING CATEGORY CODE 


Category Equipment Example 
A Manual methods only Paper and pencil 
B Adding machine - 
c Desk calculator Friden, Marchant, Monroe 
D Desk computer TEM 610, IBM 1620 
E Punched card sorter and tabulator - 
F Calculating punch TEM 602. IBM 604, IBM 607 
G Medium scale electronic computer TEM 650 
H Large scale electronic computer IEM 700 series or larger 
I Other (specify) Includes X-Y plotters and 
analog computers 
x Would not attempt within limitations of time and available 
equipment. 
Amount of Date Ne 10 50 500 5000 
| 7] 7 | 7 | oH 7 |x 
Average 
Median 
Mode 
Percentiles 
Range 


Table of 

Individual Frequencies 
‘Table of 

Grouped Frequencies 
Table of 

Cumilative Frequencies 


| 
| 
| le 
| 
| 7 
| 


Metals Technical Committee - American Society for 
mt Used to Process Data in Studies 
QUESTIONNAIRE - PAGE C - CORRELATION AND REGRESSION 


Survey of Equi 


INDICATE PRINCIPAL EQUIPMENT USED IN EACH BOX USING THE EQUIPMENT CATEGORY OODE 


EXHIBIT I (continued) 


PHILADELPHIA, PENNSYLVANIA 


ality Control 
Statistical Methods 


Amount of Data Ne 


10 


Number of Independent 
Variables 


Answer Required 


in n days 2+ 


Linear Regression 


Linear Correlation 
Coefficient 


Multiple Linear 
Regression 


Cross Product 
Extension 


Multiple Correlation 
Coefficient 


| 


Partial Correlation 
Coefficient 


Beta Coefficients 


| 


Non-linear Regression 


Non-linear Correlation 
Coefficient 


Amount of Date Ne 


Number of Independent 
Variables 


10 


10 


Ansver Required 
in n days ae 


Linear Regression 


Linear Correlation 
Coefficient 


Multiple Linear 


Cross Product 
Extension 


Multiple Correlation 
Coefficient 


Partial Correlation 
Coefficient 


Beta Coefficients 


Non-linear Regressions 


ion-linear Correlation 


Coefficient 
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EXHIBIT I (continued) 


Metals Technical Committee - American Society for Quality Control 


Survey of 


uipment Used to Process Data in Studies lo: Statistical Methods 


QUESTIONNAIRE - PAGE D - ANALYSIS OF VARIANCE 


INDICATE PRINCIPAL EQUIPMENT USED IN EACH BOX USING THE FOLLOWING CATEGORY CODE 


Category wipment Example 
A Manual methods only Paper and pencil 
B Adding machine - 
c Desk calculator Friden, Marchant, Monroe 
D Desk computer TEM 610, IBM 1620 
E Punched card sorter and tabulator - 
F Calculating punch IBM 602, IBM 604, IBM 607 
G Medium scale electronic computer TEM 650 
H Large scale electronic computer IBM 700 series or larger 
I Other (specify) Includes X-Y plotters and 
analog computers 
« Would not attempt within limitations of time and available 
equipment. 
No. of Replicates 2 5 10 25 


Ansver Required 


in days 


Single Criterion 


Double Criteria 


3 to 5 Criteria 


Over 5 Criteria 


Co-variance 


Unweighted Means 


Unweighted Means 


2x2x2 


Weighted Means 


322 
| 


EXHIBIT II 


“PHILADELPHIA, PENNSYLVANIA 


SUMMARY OF REPLIES - DATA PROCESSING EQUIPMENT AVAILABLE 


Programming 
Bquipment Availability Availability Sources 
FIGURES 
IN 
BOXES 2 
INDICATE 4 3 
NUMBER 
or [83] 3 
REPLIES (33) 
213.13 
i 3 
c 
(Refer to Exhibit II 3 2123183 2 < x > s 2 
page 2 
for Examples) 
Manual Methods Only of o 5] 7118 6] 1 5 
Adding Machine 2157 12] 1] 0 | 6 1 3 
Desk Calculator sinieleile 3] 35 | 13 | 2 5 
Desk Computer 4o 7 3 8] 6 13 8 5 3 
Punched Card Sorter 
and Tabulator 3 | 9] 1 12} 7 
Calculating Punch 5 | 5] 4] 8] & F210 fio ff is | 6 | 7] & 
is] 6] a] 16] 6 | 36 Jas 2 | 2 | 23 | a7 
Large Scale 
Electronic © ° 5 19 | 19 8 16 12 19 18 10 
Othe 4 
-- 3] 2] 7]/off3 | 6] a] e 


= 
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EXHIBIT II (continued) 


SUMMARY OF REPLIES - DISTRIBUTION STATISTICS 


Category Equipment Example 

A Manual methods only Paper and pencil 

B Adding machine -- 

c Desk calculator Friden, Marchant, Monroe 

D Desk computer IBM 610, IBM 1620 

E Punched card sorter and tabulator -- 

F Calculating punch IBM 602, IBM 604, IBM 607 

G Medium scale electronic computer IBM 650 

H Large scale electronic computer IBM 700 series or larger 

I Other (specify) Includes X-Y plotters and 

analog computers 
x Would not attempt within limitations of time and available 
equipment. 
None No answer received. 
HEADING SYMBOLS 
N - amount of data 
n - answer required in n days 
AVERAGE 

N 10 50 500 5000 
n= 1 7 x 1 7 30 1 7 x 1 7 3x 
A 22 17 | 17 6 3 3 ie} 
B 6 8] 6 8 8 7 8 8 7 3 3 e 
c uy 42 | 4e 56 50 53 28 32 31 10 13 12 
D ie) 1 3 1 1 1 1 1 1 3 5 5 
E oO 1 1 6 6 5 5 8 12 
1 1 1 3 8 8 6 9 
G 5 8 7 4 12 14 
x 14 3 3 30 11 7 
NONE 4 8 | 10 4 12 10 8 6 9 11 9 8 


4 
324 
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EXHIBIT II (continued) 


MEDIA 
| 
fe »| «| » 
jo fof pot o| 
fe of of of of of ef 5] 
of of of of of ef ef 
of of of of of of of of 
| 
Poof of of of of 2] of | 
fof of of of of of ef 
of of of of of of of of 
| 
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EXHIBIT II (continued) 


PERCENTILES 

N= lo 50 500 5000 

n= 1 7 30 7 1 7 30 1 7 x» 1 7  ) 

c 32 33 | 32 43 19 26 26 7 
D 1 2i2 2fe 2 2 2 2 3 34-5 
3 1 8 tu 4 ll | 13 
Oo] o i 1 4 5 5 
G o 3 6 6 4 8 | lo 
o 2 2 2 3 5] 4& 
x 2 1 918 6 5 33 16} 1 

wong |} 14 | 16 | 18 716 Jie Jaw 14] 

RANGE 

N= 10 50 500 5000 

n= 1 7 x fl 7 30 1 7 xn 1 7 x» 

c 1} 17] 17 el} 18 | 20 f 10 | is | as 4 6] 6 
D ae 2 1 Oo}; o 
1 1] 1 6] 6 6 Fir | a3 9 19 | 20 
Oo] o 1] o 2 1 1 3 2] 
G Oo} o o 2 5 4 3 8] 9 
Oo] o o 2 2 2 3 
I Oo}; o of o Oo} o 
x Oo}; o Oo; 1 1 fu 5 4 ag 12 |} 10 

NONE 8] 12/14 fro fiz 3 11 | 12 


= 
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EXHIBIT II (continued) 
STANDARD DEVIATION 


10 


50 500 5000 
n= 1 7 xo 71 7 30 1 7 x» 1 7 3x 
15 kein 4 3 2 0 1 1 1 1 
B 2 1 2 2 1 1 1 Oo} o 
c 52 52 | 50 56} 51 53 20 30 27 8 11] 10 
D C + e 3 3 3 3 3 5 3 3 5 
zg oO ce) fe) 1 1 1 4 4 3 2 3 6 
2 2 2 6 6 6 4 8 6 
G 1 1 7 12 13 6 17 21 
H fe) 4 5 6 7 7 
x ) Oo] o 1 1 1 20 7 4 36 16 | 10 
NONE 7 10 | 13 [iS 12 ll 7 ll 13 10 3) 
SKEWNESS 
N= 10 50 500 5000 
n= 1 7 3x 21 7 x 1 x» 1 7 x» 
7 6 5 3 3 2 1 1 1 
c 4s 46 43 4h 16 24 25 7 9 
D 2 2i 2 33.3 3 3 3 ‘, 3 31 6 
0 3 5 4 2 3 4 
Oo} oO 2] 4 5 4 6 6 
0 oO 0 1 1 6 g 10 3 15 | 18 
o 2 2 3 3 5 4 
X 10 8] 8 9}] 8 9 26 13 10 38 21 | 15 
none || 12 15 | 16 13 | 15 15 15 14 14 16 14] 14 
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EXHIBIT II (continued) 


KURTOSIS 
Ne 10 50 500 5000 
n= 1 7 x 1 7 x 1 7 xn 1 7 x 
CAT. 
c ak ys $16 fo, | 26 7 9] 8 
D 2} 37/3 fg 3 | 6 
E fe) 3 4 3 2 3 
? fe) fe) fe) 2 2 2 4 5 4 4 6 6 
o] 1 1 6 9 |10 3 15 | 18 
1 2 3 4 3 5 
x 10 818 91/8 9 728 38 21 15 
4 
TABLE OF INDIVIDUAL FREQUENCIES 
N= 10 50 500 5000 
n= 1 7 x 1 7 x 1 7 x» 1 7 x” 
“af | fas las las 9| 8 
B 3 3 2 1] o 
c 16 14 15 16714 | 15 3 5] 6 
D 2,2 3 3 2 2 2 2 2] 4 
o 2 9718 | 21. | 20 io 22 | 25 
ofo ol o 0 0 0 
G 1 1 2 7 6 3 8 | 1o 
E fe) 3 3 3 4 5 5 
x 2 2] 2 Oo 416 3 2 30 7 
wong |} 10 | 12 14 [ik 4913 9 113 fas 


| 
_! 
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EXHIBIT II (continued) 


ZABLE OF GROUPED FREQUENCIES _ 
N= 10 50 500 5000 
ry 41 41] 39 41} 4o 39 14 15 14 5 7 6 
B 1 21 3 2 1 1 o 
c 15} 14] 15 18} 15 | 16 9} 15 | 14 5 71 67 
D 2 2} 2 a5. 3 3 2 2 2 2 2} 4 
o} o 1] o oF 1 |] 2 u 23] 26 
G ) Oo} o Oo} 1 1 2 6 6 3 7} 10 
Oo} oO 3 3 3 4 
x 5 2] 2 2 18 5 5 34 9 
NONE |} 12] 12] 14 9] 12 | 13 10 7 | 10 12 10} 9 
TABLE OF CUMULATIVE FREQUENCIES 
N= 10 50 500 5000 
37 | 35] 36 27 | uf] u 3 5] 4 
B 2 3] 2 at 4 3 1 1 1 1 is. 3 
c 19} 18] 19 29] 26 | 27 § 12 | 18 | 18 6 Ti 
D 2 21 e2 2 2 2 2 2 2] 4& 
o 2} 1 16 | 18 | 17 20 20] 22 
Oo} o o 1 2 2 1 2 
G o o} 21 1 2 8 7 3 13 
o o 3 3 3 4 5] 5 
x 4 2 19 5 5 34 9 
mar} 22 | 1 | 23 10} 13 | 12 10 8 | 10 22 lo} 9 
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EXHIBIT II (continued) 


SUMMARY OF REPLIES - CORRELATION AND REGRESSION 


Category Equipment Example 
A Manual methods only Paper and pencil 
B Adding machine -- 
c Desk calculator Friden, Marchant, Monroe 
D Desk computer IBM 610, IBM 1620 
E Punched card sorter and tabulator -- 
F Calculating punch IBM 602, IBM 604, IBM 607 
G Medium scale electronic computer IBM 650 
H Large scale electronic computer IBM 700 series or larger 
I Other (specify) Includes X-Y plotters and 
analog computers 
xX Would not attempt within limitations of time and available 
equipment. 
None No answer received. 
HEADING SYMBOLS 
N - amount of data 
Vv - number of independent variables 
n - answer required in n days 
LINEAR REGRESSION 
Ne 10 50 500 5000 
Ve= 1 1 1 1 
i 7 T 30 1 7 1 7 xn 
CAT. 
B 2 2 1 2 3 1 
c 54 49 | 52 | | 4B 1k | 19 | 17 5 7 8 
D 1 2 2 3 3 3 3 5 > 1 1 3 
F ° i?) ° 1 2 2 2 5 4 5 6 6 
G ° A 1 2 3 4 4 18 23 4 22 31 
H 1 1 1 2 3 3 5 7 7 5 7 8 
x 5 37 7 8 42 | 12 
NONE, 8 14 13 10 10 | 13 11 14 11 14 20 9 
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EXHIBIT II (continued) 
LINEAR REGRESSION (continued) 

Ns 50 500 5000 50 

Ve 5 10 
i 7 T 30} 1 7 1 7 3 
wt 1 1 fe) fe) 1 1 2 
c 9 17 17 4 5 7 2 2 3 4 6 6 
_D 4 5 + 3 3 5 1 1 3 4 5 7 
F 2 2 2 2 5 2 2 4 4 2 2 2 
G 8 | 22] 7] 2h | 30 6} a2 | 337 10] 2 | 27 
H 5 6 6 5 7 G 4 8 9g 5 6 6 
x 33 9 6 ft} 41 16 8 I} 43 2k 8 34 16 10 
wone || 15 we] 15 | 35 17 15 15 16 15 16 

ae 500 5000 500 9000 

v= 10 10 25 25 
1 7 1 30} 1 30 | 1 7 x 
wr o 0 0 0 0 0 
c 1 1 5 2 2 4 2 2 2 2 2 2 
D 3 3 5 1 1 1 2 2 2 1 1 1 
F 2 3 3 2 1 1 1 1 
G 7 28 ‘Ly 4 18 27 5 14 2k 4 12 23 
4 8 ll 5 14 6 10 13 
x 45 19 9 45 28 15 45 32 19 45 34 21 
wore | 15 | 16 |] 15 17 | 5 | 16 |} 16 | 16 | 15} 17] 16] 16 
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EXHIBIT II (continued) 
LINEAR CORRELATION COEFFICIENT 


Nes 10 50 500 5000 

1 1 1 

7 x 1 30 1 7 x] 1 7 x» 

x 4] 3 3 O°} Of of of of of of of o 
B 1 1 1 1 2 1 2 
Cc 58 | 53H] 49] SO} 13] 16 16 5 5 6 

2) 3] 3] sf si si sf al al 
F 1 2 2 2 5 4 5 7 6 
G 1 3 4 5 18] 4] 23] 31 
E 1 1 1 1 3 3 5 7 7 5 7 8 
1 1 1 5 36 | 10 87 41] 19] 10 

Wong} 10) 15) 14/13] 12] wi} a3] a5] 15] 1] 
Ne 50 500 5000 50 

nelle 5 5 5 10 

Ber 3 7 x 1 7 30 1 7 3 7 1 7 xn 

c 9] 18] 17 3 3 6 2 2 3 4 7 7 
D 4 5 4 3 3 5 1 1 5 4 5 7 
F 2 2 2 2 5 2 3 5 4 2 2 2 
G 8 22] 8} 23] 3 6 10] | 27 
H 5 6 6 5 | 9 4 8 9 5 6 6 
x 35 | 8 be | 18 9] | 28] 37] 19] 20 

NONE}; 13 12 15 13 16 | 12] 


| | 
| 
| 
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EXHIBIT II (continued) 
LINEAR CORRELATION COEFFICIENT (continued) 


Ns 500 5000 500 5000 

Ve= 10 10 25 25 

30 1 T 30 1 30 21 

c 1 1 5 1 1 3 1 1 2 1 1 2 

D 3 3 5 1 1 1 2 2 2 1 1 1 
F 1 2 3 3 2 1 1 1 1 
G Ti @ 34 18 26 5 14h 12] 
H 4 7 8 4 9 11 5 9] 14 6 10 13 
x 47 | 22] 11 |} 49 | 32 | 49 | 36) 227 49 | 39 | 23 
13 | 14 13 12 | 13 13] 13 | 127 12) 13 

MULTIPLE LINEAR REGRESSION 
ae 50 900 9000 50 


333 
| | 
| 
| 
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| | | 
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i 
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rf of of of of of of 
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EXHIBIT II (continued) 
MULTIPLE LINEAR REGRESSION (continued) 


Ns 500 5000 500 5000 
v= 10 10 25 25 
ra} of of af of of of of of of of of o 
c 1 1 4 1 1 2 1 1 1 1 3 1 
D 3 4 6 1 1 1 2 2 2 1 1 1 
1 1 1 1 1 1 1 1 
G 18 30 5 25 12] 2b 
H 4 7 9 4 9 11 5 9 14 6 10 | 13 
» 4 47 | aL 8 I} 48 31 16 47 35 a1 46 37 | 22 
13 | 14 | 13 16 | 13 | 14 15 | 1h] 17] 14] 15 
CROSS PRODUCT EXTENSION 
| Ne 10 50 500 5000 
| v= 1 1 1 1 
1 1 1 1 1 1 
c 35 33 33 30 31 30 9 8 8 4 3 2 
D 1 1 1 2 4 4 1 1 3 
F fo) 1 1 1 1 1 fe) 5 6 2 6 8 
G 1 1 1 2 8} 10 2] 12] 
q 1 1 1 1 2 2 3 6 5 3 4 5 
x 4 4 4 6 3 3 23 7 6 26 13 6 
NONE}; 34 35 35 35 36 | 37 37 4 36 38 37 37 
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EXHIBIT II (continued) 


CROSS PRODUCT EXTENSION (continued) 


| Ne 5 500 3000 50 
| v= 5 5 5 10 
| Sen 3 x} T x 7 1 x» 
c 73m tS 3 4 3 2 3 3 3 ~ 5 
D 3 3 3 3 4 6 1 1 ~ 3 5 7 
F 3 4 4 3 6 6 3 6 4 2 3 2 
G 6 | 17] 18 5} 16] a ob 7/19 | a 
Eq 5 6 7 5 6 8 4 7 9 4 6 7 
x 32 [22 | | 19 | 12 | | | 
WOME!) 20 | 21 | 22 1118 | 19 | 2 |I 18 17 | 18 J 19 | 19 | 19 
| 300 5000 500 3000 
| V«= 10 10 25 25 
| Pen 7 1 1 7 1 7 x 
Cc 1 1 3 1 1 2 1 1 1 1 1 1 
D 3 4 6 1 1 1 2 2 2 1 1 1 
F 1 3 1 3 3 3 1 2 1 1 2 1 
G 5 | | 27 13 a 7 | 2 8 | 19 
K 4 6 7 4 8 | 10 5 9 | le 6 | 10 | 13 
x 43] 23/13 | 32 | 20 | 35 | 36 | 22 
mone}; 19 | 19 | 19 #119 | 19 | 19 flig | 2 | 19 19 | 18 | 19 
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EXHIBIT II (continued) 
MULTIPLE CORRELATION COEFFICIENT 


N= 50 900 5000 50 
Vs 5 5 5 10 
1 30 T 2 | 7 x» 
c 10 16 16 3 3 6 1 2 3 4 7 7 
D 4 4 4 3 + 6 1 1 4 4 6 8 
F 2 2 1 2 5 2 4 6 4 2 3 2 
G 7 al 25 7 2h 31 5 19 35 10 25 2 
E 5 6 7 5 7 9 4 8 | 10 5 6 8 
x 35 | 13 8 |} 43 | 18 9 || 49 | 30 | 10} 39 | 18] 10 
13 | 13 | 15 23 | 24 | 13 12 | 10 | 
N= 500 5000 500 5000 
V«= 10 10 25 25 
i 7 30 2 7 1 7 x 
c 1 1 5 1 1 3 z 2 3 1 2 3 
D 3 4 6 1 1 1 2 2 2 1 1 1 
F 2 3 1 3 4 3 2 2 1 2 2 1 
G 7| 2] 36 17 | 2 5] 25 1] 
7 9 9} 1 5 9 6] 10] 13 
x 48 | 22 49 32 | 35] at 2 
11 | 13 | 10 14 | 12 | 12 12 | 12] 119 13] 12] 12 
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EXHIBIT II (continued) 
PARTIAL CORRELATION COEFFICIENT 
N= 50 500 3000 
Ves 5 5 5 10 
4 T 30 1 7 1 7 
Pet of of of of of of of of of of of 3 
Cc: 8 |] 13] 15 2 4 4 1 2 3 3 5 6 
D 4) 3] &] 2] at} uf af 6] 6 
F 2 2 1 2 5 2 3 5 3 2 2 1 
G 7] 22 6 2 | 2 5] 18 30 9] 2] 
H 4 5 6 4 5 7 3 6 8 4 5 7 
x 36 | 21 12 || 49 15 21 15 
NONE}; 15 | 16 17 15 16 17 14 | 13 | 139 15 14 
500 5000 500 3009 
Ve= 10 10 25 25 
Ser i T T 21 7 1 7 30 
c 1 1 4 1 1 3 1 2 3 1 2 3 
D 3 4 6 1 1 1 2 2 2 1 1 1 
F 1 2 3 3 2 1 1 1 1 
G 6 | 31 4 | 17 | 26 4 13 4] 12 23 
H 3 5 7 3 7 9 4 8 ll 5 9 12 
x 46} 25 | 14 49 | 33 | 20 | 35 Sin 
WOME) 124 15 14 | 15 | a5 | a5 as | ae 15 
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EXHIBIT II (continued) 


BETA COEFFICIENTS 


500 5000 50 
ve > 5 5 10 
oe} 1 7 x 1 T 30 1 7 30} 21 7 x 
Pre of of of of of 
c 14 2 3 1 3 3 3 5 
D 4 4 4 3 4 6 1 2 5 4 6 8 
: 2 2 1 2 5 2 3 5 3 2 2 1 
G 7} 19 | 22 Ti 16] 2 a] 23 
q 4 5 6 4 5 7 3 6 8 4 5 7 
x 35 | 16] 12 | 43] a 13 || 48 31 | 16 39 | 22] 17 
WOME), 15 | 15 | 17 |] 15 | 16 | 16 16 | 13 | a5 a5 | 15 | 15 
N= 300 5000 500 5000 
v= 10 10 25 25 
3 7 x | 1 T 1 7 30} 1 7 3x 
c 1 1 4 1 1 2 1 2 2 1 2 2 
D 3 4 6 1 1 1 2 2 2 1 1 1 
F 1 2 3 3 2 1 1 1 1 
G 6 | 23] 30 3] 16] 2 12] 23 3 9 | 22 
H 3 5 7 3 T 9 ~ 8] 12 5 9] 12 
x 47 | 25 15 48 33 | a ko 35 22, 48 36 23 
HOKE 15 14 17] 15 |} 16 | 16] 17; 16] 16 


| 
| 
| 
| 
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EXHIBIT II (continued) 
NON-LINEAR REGRESSION 


Ne 10 90 300 9000 

ve 1 1 1 1 

pei 2 | 7 | 7 | 12 | 7] 7] 

7] 6] 69 & 3 3 of of of o 
B 8 1 1 Of Of of of o 
c 35 | | 32] 33 uj un 4 
Dp. 2 1 1 3 3 3 3 5 6 2 2 5 
E Oo; Of of of} of of 12 1} oO]; o 
F of Of of off o 2} 2] o 2} 2 
G 3 3 3 7 7 4} 16] 19 4] 18] 22 
1 1 1 2] 3 3} 7] 6 7] 7 
I Of Of Of Of Of of of of o 
x il 6 6 |] 15 7 > |} 33 7 8] 39] 18] 13 
WOME); 19 | 22 | 22 19 | | 21 |} | 27] 23 23] a] 20 
N= 90 900 9000 

i 30 1 T 21 1 x 

1} 1 Of; Of of o 1 1 1 
B Oo; Of Of Of Of of of of of o 
c 5 | 10 8 2 2 “ 1 1 2 2 3 4 
D 4 5 7TH 3] 4] 6 1 1 4 4 5 7 
E Oo} 1] off o 1 Of Of} O} Of Of of o 
F 3 2] 5 2} 1 3 1 2} 2 1 
G 8] 18] a 7] 18 | 25 4] 1b) 27 8] 19] 23 
6 7 4 7/ 9 10] 6 7 
I Oo} Of Of Of Of of of of of of o 
x 35} 16 | 12} 2 | 12H 38] 22] a5 
17 17 | 19 17! 19 | 18 18] 17] 18] 18 


— 
a 
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EXHIBIT II (continued) 
NON-LINEAR REGRESSION (continued) 


N= 500 5000 500 5000 

V-= 10 10 25 25 

n=? | 2] 7] 2 | 7] 

c 1 1 2 1 1 1 1 1 1 1 1 1 
D 1 2 4 1 1 1 2 2 2 1 1 1 
F 1 2i 1 1 1 2 1 1 2 1 
G 6 | 18 27 | 12 | 22 5} 4] 10] 19 
5 9} 12 u | 15 4 9 | 5} 10] 13 


NONE); 17 19 18 17 | 17 17 16 17 16 17 16 17 


NON-LINEAR CORRELATION COEFFICIENT 


N= 10 50 500 5000 
Ve 1 1 1 1 
1 1 30} 1 7 30 1 7 xn 


D 1 1 1 3 3 3 2 
F fe) fe) te) te) 3 3 2 2 


G 1 3 3 $i 17] a 


7 
| | 
fof of of of of of of of of of of 
| 
[x of off of of of of of of of of | 


; PHILADELPHIA, PENNSYLVANIA 


EXHIBIT II (continued) 
NON-LINEAR CORRELATION COEFFICIENT (continued) 


N= 30 500 5000 50 


5 
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EXHIBIT II (continued) 


SUMMARY OF REPLIES - ANALYSIS OF VARIANCE 


Category uipment Example 
A Manual methods only Paper and pencil 
B Adding machine 
c Desk calculator Friden, Marchant, Monroe 
D Desk computer IBM 610, IBM 1620 
E Punched card sorter and tabulator -- 
F Calculating punch IBM 602, IBM 604, IBM 607 
G Medium scale electronic computer IBM 650 
H Large scale electronic computer IBM 700 series or larger 
I Other (specify) Includes X-Y plotters and 
analog computers 
xX Would not attempt within limitations of time and available 
equipment. 
None No reply received. 
HEADING SYMBOLS 
R - number of replicates 
n - answer required in n days 
SINGLE CRITERION 
R= 2 5 10 25 
n= 1 7 3x 7 30 1 i 7 x 
CaT. 
A 9 8 8 3 2 2 2 1 1 1 te) ° 
c 4h 46 39] 42 | 36 § 28 | 30 | 32 23 22 | 2h 
D i?) 1?) ie) 2 2 4 4 4 5 3 2 4 
1 1 2 2 1 2 fe) 2 2 
1 2] 3 3 2 3 3 5] 
x 8 6 6 14 7 6 23 12 8 27 17 | 1le2 
NONE |} 14 13 | 15 16 | 16 18 17 19 18 17 20 | 19 
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EXHIBIT II (continued) 
DOUBLE CRITERIA 


5 


10 


25 


2 


; 343 
——— 
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EXHIBIT II (continued) 


OVER 5 CRITERIA 
Re 2 5 10 25 
c 28 34] 36 20} 23 | 25 9 17 | 19 6 9] 12 
D 1 2 e 4} 7 7 4 5 6 3 3 5 
of} 3] 3 oj & | o | 3 | 34 0 ij 
G 2 2] e 2] 5 5 8 | 12 7 12 | 18 
3 3 3 4 5 5 6 7 5 7 
x 23 12 | 10 27 | 13 10 35 17 9 37 23 | 13 
NONE || 17 | 18 | 19 18117 |19 17 |19 | 20 | 17 20 | 19 
CO-VARIANCE 
R= 2 5 10 25 
1 o 1} o 1 1 o 
B 1j o 1 1 1] o 
c 37 35 | 37 26 | 25 26 11 16 18 8 lj lu 
D 1] 6} 6h & 5 3 2] 4 
3] 3 4 3 3 3 0 2 
G 1 3 3 1 4 6 4 8 1 6 13 | 17 
q 1 1 1 2 3 2 3 4 4 3 6] 6 
x 10] 9 al 9 32 16 | 12 34 19} 13 
NOME |} 21 | 22 | 22 21} 21 | 23 21 | 23 | 23 § 21 23 | 23 
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EXHIBIT II (continued) 
UNWEIGHTED MEANS 


1 1 1 1 | 
fas | Poo fos fas fro fos | | 
UNWEIGHTED MEANS 2x2x2 
of of of ofa} of of of o| 
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EXHIBIT II (continued) 


WEIGHTED MEANS 


R= 2 5 10 25 

i | 7/992 | 7 | | 7] 2 | 7 

B o} 1 1 1} o 

c 32 | 32] 34 20] 20 | 23 12 11 | 16 20 lo | 13 
D 1 ; | 1 4 6 6 4 4 5 3 2 4 

of o o 1 1 of o 

3] 3 4 3 3 3 2] 1 

G 2] e2 6 2 6 7 3 8} 11 

1 2 21 3 2 3 5 3 6] 6 

x 15 10] 7 21 | 10 7 | 16 | 10 29 18 | 12 

NONE |} 26 | 26 | 28 a7 | 29 29 27 29 | 29 
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EXHIBIT III 


MOST FREQUENTLY RECEIVED REPLY 


Manual methods only 
Adding machine 

Desk calculator 
Desk computer 


Punched card sorter 
and tabulator 


Calculating punch 
Medium scale electronic 
computer 


Large scale electronic 
computer 


Other 


nt Availabilit 


Immediately available 
Immediately available 
Immediately available 
Not available 


Available on request 
within plant 


Available on request 
within plant 


Available on request 
within plant 


Not available 


Available on request within 
corporation outside plant 


Programming Availability 


Manual methods only 
Adding machines 
Desk calculator 
Desk computer 
Punched card sorter 
and tabulator 


Calculating punch 


Medium scale electronic 
computer 


Large scale electronic 
computer 


Other 


Most programs required 
available 


Most programs required 
available 


Most programs required 
available 


No prepared programs 
available 


Most programs required 
available 


Tie for all classifications 


A few basic programs 
available 


Most programs required 
available 


Most programs required 
available 


2k 


18 


18 


each 
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EXHIBIT III (continued) 


Programming Sources 


Manual methods only Write ow 30 
Adding machine Write own 31 
Desk calculator Write own 35 
Desk computer Manufacturer 9 
Punch card sorter Write ow 17 


and tabulator 


Calculating punch Write own 15 

Medium scale electronic Write own 26 

computer Manufacturer 26 

Large scale electronic Manufacturer 19 . 
computer 


Other Write owm 6 
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EXHIBIT III (continued) 


MOST Y RECEIVED REPLY 
Distribution Statistics 


LETTERS IN BOX INDICATE EQUIPMENT CATEGORY, FIGURES INDICATE NUMBER OF REPLIES 


Category uipment Example 
A Manual methods only Paper and pencil 
B Adding machine ° 
c Desk calculator Friden, Marchant, Monroe 
D Desk computer IBM 610, IBM 1620 
E Punched card sorter and tabulator - 
F Calculating punch IBM 602. IBM 604, IM 607 
G Medium scale electronic computer IBM 650 
H Large scale electronic computer IBM 700 series or larger 
I Other (specify) Includes X-Y plotters and 
analog computers 
x Would not attempt within limitations of time and available 
equipment. 
Amount of Data Ne 10 50 500 5000 
Ansver Required 
in n days 71/909 f 217 | oH 1] 7 
Average c c c c c c c c c x c G 
| 56 | 50 53. 268 | 32 31 14 
Median A A a A A A c c x E E 
4S xX 39 34 15 15 19 
A A A A A A A E A x E E 
Mode 
43 38 38 33 31 31 18 | 17 17 31 | 20 22 
Percentiles fe jc fe fe fc x Ix 
32 | 33 | | 43 19 | 26 33 | 16 1k 
Range A A A A A A A A x A E 
= + 26 25 _20 
C-X jc ic x G G 
Standard Deviation se | se | of 6 | | | 9 | a 
c c c iC ic ic x c ic A x G 
c c c IC iC ic x c ic x x G 
Kurtosis 
45 46 43 28 26 3% 18 
Table of A A A A A E x E 
Individual Frequencies 45 uy 42 ft 41 39 c's} 18 21 20 3x | 22 _ 25 
rouped Frequencies 41 42 18 22 2} | 23 
Table of A A A A-C IA A x C-E jc x E E 
Cumilative Frequencies 37 | 35] 36 2 | 27 | 19 | 16 | 16 34] 2 22 


a 
é 
¥ 
a 


PHILADELPHIA, PENNSYLVANIA 


EXHIBIT III (continued) 


MOST FREQUENTLY RECEIVED REPLY 


Analysis of Variance 
LETTERS IN BOX INDICATE EQUIPMENT CATEGORY, FIGURES INDICATE NUMBER OF REPLIES 


Category Equipment Example 
A Manual methods only Paper and pencil 
B Adding machine * 
c Desk calculator Friden, Marchant, Monroe 
D Desk computer TEM 610, IBM 1620 
E Punched card sorter and tabulator - 
F Calculating punch IBM 602, IBM 604, IBM 607 
G Medium ecale electronic computer TEM 650 
H Large scale electronic computer IBM 700 series or larger 
I Other (specify) Includes X-Y plotters and 
analog computers 
ne Would not attempt within limitations of time and availavle 
equipment. 
No. of Replicates 2 5 10 25 
Required a= f} i} 2] oat 7] 
Over Criteria [x6 | “2s | “35 [a9 [2 | 23_| 29 
| “es | | 2s [es [es | es 
Vaweignted Means | ‘34 27 | 27 | |x |] |» 
Vaveignted Means 27 | 27 | ol] 2 |x ‘2 |» 
Weighted Means x2 | x “28 | “27 ¥ "29 
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MECHANIZED RETRIEVAL OF SCIFNTIFIC INFORMATION 


Vida Grace Hiidyard 
Staff Consultant to the Director 
Engineering Services 
Remington Rand UNIVAC 
Division of Sperry Rand Corporation 
St. Paul, Minnesota 


INTRODUCTION 

"File It to Find It," was the title of the lead article in the September 1959 is- 
sue of INDUSTRIAL OMUALITY CONTROL, In it, Ervin F. Taylor, Chairman of the Bibliogra- 
phy Committee, described the American Society for Quality Control literature classifi- 
cation scheme, "Classification," he said, "is not for the purpose of merely filing. 
... Classification is for the purpose of finding.” In an Editorial Note, Mason Wescott 
added that this system was developed as “a solution to a major problem - aiding the 
ASC member in locating the literature he wants from the vast quantities supplied by 
the ASOC." 


The classification scheme, which resembles the Dewey Decimal System of library 
cataloging, is a closed-end one, of three major categories: methodology or techniques 
are limited to a maximum of 1000 classifications; functional classifications are 
limited to 100; and industry and business ciassifications cannot exceed 1000. - 


For a relatively homogeneous collection of literature, in which no very great 
depth of indexing is required to locate an article or articles on a particular subject, 
such a classification scheme works satisfactorily. For a collection of many thousands 
of documents of heterogeneous subject-matter, an open-ended system - that is, one with 
no limit on the number or type of index terms - is more suitable. This method makes 
it unnecessary to force a document into pre-existing categories. 


During the last 15 years, many individuals and organizations have been concerned 
with the development of Information Storage and Retrieval systems. Both the Senate 
and the House of Representatives of the United States Congress have conducted investi- 
gations and have made appropriations for such work. Some Government agencies which 
have been doing or contracting work, or making investigations of systems, are the 
Library of Congress, the Central Intelligence Agency, the Patent Office, the National 
Pureau of Standards, and the National Science Foundation, to name just a few, Many 
private organizations and industries have also been evolving or investigating systems, 


Efforts have heen of two general types: those interested in building machines, 
and those concerned with developing theory. Conventionally trained librarians have 
for the most part, been hostile to hoth groups, usually cleiming thet “library sci- 
ence” methods were already doing a sufficient job, and/or that the goals of the “infor- i 
mation scientists" were impossible to achieve. 


True, library method are effective - up to a point. Rut both librarians and 
documentalists are at the mercy of their filing systems. No amount of mechanization 
wit] retrieve anything which has been mis-filed or mis-coded, or not indexed in suf- 
ficient depth to make it retrievable on demand. ‘ 


The first published article to give serious consideration to the mechanization of 
literature searching was one in the July 1945 issue of the ATLANTIC, by Vannevar Bush, 
then head of the Office of Scientific Research and Development, The first machine for 
information storage and retrieval, called the Rapid Selector, was developed by 
Dr. Ralph R. Shaw, now at Rutgers University, then librarian of the Department of Agri- 
culture, Since that time, several other special purpose machines have been produced, 
and a number of organizations have come to use general purpose computers to store and 
retrieve information, The largest of the users of a general purpose computer is the 
Armed Services Technical Information Agency, which uses & UNIVAC Solid State 90 comput- 1 
er on its mechanized file of over 250,000 research and development reports on all 
government contracts since the end of World War II, Current growth rate of this 
collection is 30,000 documents per year. 
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Contrary to much popular opinion, there is nothing magic about electronic comput- 
ers, so that they cannot retrieve information that was not properly abstracted and 
indexed in the first place. Consequently, the more recent emphasis has shifted some- 
what from machine design to theory of indexing and retrieval. 


WHY DOES AN ORGANIZATION NEED AN INFORMATION STORAGE AND RETRIEVAL SYSTEM? 

Whenever a new project is undertaken, many problems must be resolved: Fstablish- 
ment of personnel requirements, budgets and schedules; specifications and standards, 
quality control procedures, testing programs; component and vendor selection, sub- 
contract letting: and a host of other technical questions - all requiring extensive 
research of one sort or another, Most new projects have a number of elements in com- 
mon with others currently in progress or already completed, either within one's own 
organization or elsewhere in the world. Hence FORTUNE Magazine, in an article called 
“How to Cope with Information", in the September 1960 issue, says, "Scientists and 
engineers are haunted by the fear - indeed the certainty - that they are working on 
problems that have already been solved and whose solutions have been published some- 
where." 


Investigation of a particularly knotty problem often delays work for weeks. All 
too often, after such delay, the truth of this quotation is evident. It has even 
happened that a man in the same office, whom no one thought to consult, says, “Pardon 
me, gentlemen, I couldn't help overhearing. We faced this same problem a couple of 
years ago, on Project X. I was involved in it, and wrote up the final report." All 
too frequently he may add, "If we can just find it.” 


FOPTUNE qyotes an estimated minimum of 10 percent of the nation's public and 
private $12.5 billion research and development budget going for pointless duplication 
of effort. Unfortunately, such duplication is often faster and even cheaper than 
finding the appropriate documents. Technical information is being generated at such 
a stupendous rate that it is extremely difficult to keep up with what one needs to 
know. 


The WALL STRFET JOURNAL of December 20, 1960, says, under the title "Fishing for 
Facts," "“Fvery 24 hours enough technical papers are turned out around the globe to 
fill seven sets of the 24-volume Encyclopedia Britannica." FORTUNE (quoted above) says, 
“Mankind is learning things so fast that it's a problem how to store information so it 
can be found when needed. Not finding it costs the U.S. over $1 billion a year.” 


HOW IS AN INFORMATION STORAGE SYSTEM DFVFLOPED? 

The three most prominent indexing philosophies which have been developed are the 
Uniterm system of Mortimer Taube, the descriptor concept of Calvin Mooers, and the 
“role indicator" method of Western Reserve University. Taube advocates using for 
indexing purposes the key words which actually appear in the document itself. Mooers' 
System uses as “descriptors” the basic concepts with which the document is concerned, 
Western Reserve goes a step further than either of. these, conveying the relationship 
between terms and ideas by means of "role indicators." Another philosophy, particular- 
ly valuable for certain types of subject matter, embodies hierarchical classifications. 
(Example: airedale, terrier, dog, animal, mammal, vertebrate.) For most collections, 
some comhinatior of uniterms, descriptors and role indicators is probably the most 
nearly idea}. For instance, a combination of the two words "cooling" and "water" 
requires a “role indicator” to disclose whether a document deals with the use of water 
as a cooling agent, or the process of the cooling of water. 


Once a suitable indexing plan has been devised, there are many different ways in 
which’ the reference information from technical documents may be stored for effective 
retrieval. Some of these are cross-referenced files in folders; hand-entries in note- 
books or on cards; edge-notched cards which can be sorted manually with a “needle"; 
“peekaboo" cards; punched cards with or without microfilm inserts, manipulated by 
either tabulating equipment or computer; or magnetic tape for computer processing. The 
choice of method will be larcely influenced by the size and complexity of the document 
collection, and the depth of indexing required for efficient retrieval. 


It must be re-emphasized that the first consideration is not the mechanics of how 
the references are to be stored and the information retrieved, but the gathering of 
pertinent documents into a central locetion, in such a form that they may be processed 
by .some means for most efficacious use. The processing requires, for each document, 
the appropriate bibliographical facts; a relevant, informational type abstract; and the 
selectior of the key terms, or ‘escriptors, by which the subject matter should be 
indexed. 


Te 
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In any organization, the logical first step in implementing an Information Storage 
and Retrieval Program is the issuance of a Management Procedure, establishing the 
existence of an Information Center and outlining the method by which documents will be 
submitted and processed for inclusion in the system. Certainly some discriminatory 
efforts will have to be expended, so that only documents of future reference value will 
he included, Likewise, measures must be instigated for obtaining documents which are 
pertinent. 


A cover sheet for each article is desirable, listing its security classification, 
contract number applicable, corporate author, project author, individual author(s), 
date of the report, and the file lovation, as well as the title or subject of the docu- 
ment. Next on the form is an abstract, as short as possible, but of sufficient length 
to contain the siqnificant information described in the referenced document, Finally, 
the key terms or “descriptors” of the document are indicated. These may be underlined 
in the text of the abstract, supplemented, at the bottom of the form, by a list of any 
terms implied by the content of the document or required for effectual indexing. Also 
in this list, any abbreviations used should be written out in full. 


Authors of internally generated documents may be asked to fill out the cover sheet 
for each document they submit. For externally generated documents, such will necessar- 
ily be accomplished by personnel of the Information Center. To be done properly, this 
requires personnel who have cocnizance of the subject matter. Clerical help cannot be 
oiven this assignment. 


In the Center, the document is assigned an accession number, and the descriptors 
are “edited.” Some judgment must be exercised as to whether they are too many or too 
few, too general or too specific, or consistent with the indexing system. Next, a 
Thesaurus is consulted, when appropriate, to determine which of several synonymous 
descriptors is the one being used in the "standard" index, and which ones should be 

cross-referenced, This is primarily to insure that a requestor will get all available 
information pertaining to the subject of his question, regardless of whether he uses 

the exact words in the index, or similar ones. Unfortunately, there is no such thing 

as a standard thesaurus of all technical subjects, so that each Center is obliged to 
develop its own in accordance with the subject matter with which it is dealing. There 
are, however, rany sources of assistance in such development, and their use is recommend- 
ed. 


Cover sheet, descriptor selection, and accession number assignment having been 
completed, the information from the document is ready to be stored for future retrieval. 


There are two basically different methods of storing information, In one, under 
suitable identification of a specific document, all descriptive terms pertaining to 
that document are listed. In other words, storage is document by document. This method 
is the simpler for inpnt purposes. Put the output is much more complex and time consum- 
ing, under most circumstances. Supposing, for example, someone requests all the docu- 
ments pertaining to wire-wrap connectors. To fulfill this request it is necessary to 
look at the terms listed under every document in the collection, to see which ones are 
actually identified as containing information on wire-wrap connectors. 


The other method, referred to as an inverted matrix, uses the descriptors as 
headers, and lists under each descriptive term all the documents in which this term is 
discussed. het is, storage is descriptor by descriptor. Input by this method is more 
time consuming and slightly more complex, but output (customer service) is generally 
much simpler and faster. 


Nepending upon the storage system selected, terms may be recorded in code or in 
straight English words. A dictionary for translation purposes is required if terms are 
coded. With a computer, this is quite simple to use. As a document collection builds, 
the rate of qrewth of the descriptor list far exceeds that of the number of documents, 
When the collection reaches somewhere between 1,900 and 3,000 documents, descriptors 
will probably not be added so rapidly - there will be much greater usage of already- 
estahlished ones - but it is inconceivable that the time will ever be reached when no 
new descriptive terms are required. 


If one is using a document-by-document storage system, to enter a new document 
one simply records the document accession number in order, with all pertinent terms 
under it. These will include bibliographical information, as well as technical terms. 
If one uses the inverted matrix method, the document accession number will be listed 
under each of the appropriate descriptive terms, including the bibliographical ones, A 
simple example of this type system is the “peekaboo" card, in which hole position 
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indicates document accession number. One would assemble all the appropriate descriptor 
cards inte one stack, and punch the number into all cards simultaneously. Numbers are 
represented hy the x and y coordinates of the hold. Thus, document number 5623 is 
represented hy a hold punched 56 spaces to the right on the x axis, and 23 spaces up 

on the y axis. When the document collection exceed the number of holes the card will 
accommodate, it is necessery to make a duplicate set of cards of another color, to take 
care of the additional document numbers. In this method, it is difficult to remove the 
accession number of a document which has become obsolete. Usinc a computer, the remov- 
al of an obsolete document number is quite simple, 


An easy way to expedite dissemination of information from the collection is to 
have all document cover sheets reproduced in compressed form in an "Abstract Book," by 
document accessior number, with copies of this book liberally dispersed in every area 
of the organization thet might have need for the technical information. How this book 
is used will be described later. 


NOW THE INFORMATION IS STORED, HOW IS IT RETRIEVED WHEN NEEDED? 

There are three methods by which information may be requested: 

1, The questior to be answered may be written out and mailed to the Center. 

2. When a question is received by telephone, it is written out by the person 
taking the message, and read back to the requestor for verification, just as 
is a telegram telephoned to Western Union. This is to insure that there is 
no misunderstanding as to how the question was stated, and what was meant. 

3. One who needs information may come to the Center in person, deliver his 
question, and perhaps wait for an answer - depending upon the method by which 
the retrieval system operates. Aqain, the question is written out for verifi- 
cation, 


As in the case of input to the system, the Thesaurus is consulted, when appropri- 
ate, to insure that the requestor will get all available information pertaining to 
the subject of his question, regardless of whether he has used the exact words in the 
index, or similar ones. 


The document collection is then searched for the accession numbers of the docu- 
ments which have in common this combination of descriptive terms, or an acceptable 
modification of it. A computer retrieval system may then actually print out the 
abstracts of the applicable documents, as is done at ASTIA. 


If abstracts are not printed out as a part of the searching system, a list of 
accession numbers of applicable documents is obtained, Next, the cover sheets (or 
appropriate entries in the Abstract Pook) are scanned to verify that al! the documents 
actually do appear to apply to the question. If there are any which do not, these are 
crossed off the list. 


Answering, like requesting, may be done in three ways: 

1. To answer a request rer-eived by mail, the list of applicable document acces- 
sion numbers, or the complete abstracts as produced by computer, is mailed to 
the requestor. 

2. If the question was received by telephone, the accession numbers are given to 
the requestor by phone. (He may also receive the written list, if he wishes.) 
Then he consults the Abstract Book at his desk. 

3. If the question was delivered in person, and the system is such that he may 
obtain a reply while he waits, the requestor may consult the appropriate 
abstracts in the Center, either in the Abstract Book or from the computer- 
produced text. 


In this last case, he requests copies of the documents he wants to see. The 
Abstract Rook (and the cover sheet for each document) tells him to file locations. 


Havine supplied the requestor with the list of accession numbers for the abstracts 
in answer to his question, the Information Storage and Retrieval Center then enters 
the following data, supplemented by follow-up information from the requestor, in the 
"Use" file: 


1, The question, The same type of question may be asked frequently. If so, the 
Center can then cive a requestor the names of others who have similar inter- 
ests, with whom he may want to get in touch, 

2. The name of the requestor, This, of course, ties in with the previous state- 


ment. It will also enable the Center to accumulate a "who is interested in 
what" register, so that at some future time it may inform such people, ahead 
of direct requests, when some new material comes in which should be of inter- 
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est to them, 

3. The descriptors. A good set of descriptors simply does not descend like manna 
from heaven. nly experience will determine what are good and what are not 
good. If there is a descriptor which has been in the index for a couple of 
years without ever having been used in a request, the chances are that it was 
a poor choice and should be eliminated. The converse is not necessarily true. 
If a descriptor has had a very high rate of usage, it may be too general, and 
should be ‘broken down into two or three more specific ores. 

4. The document numbers applicable to the question (this information obtained 
from the requestor). This serves two purposes: 1) A record of the accession 
numbers of the documents which actually provided good answers to a particular 
question will enable the Center to give improved service in the future; and 
2) It gives a clue to obsolescence. If a document has been in the system for 
several years without ever having been used for reference, the chances are 
that it should be archived. 


With an efficient Information Storage and Retrieval System, such as has been 
described, one should be able to find the documents he needs when he needs them, sub- 
ject only to the Center's acquiring the documents in the first place, Thus it should 
be possible to eliminate much costly duplication of effort, effecting substantial 
savings in both time and money. Joe Blow may still have to say, when faced with a 
problem on a new project, "Gee, I know I have read a solution to this problem some- 
where.” But instead of saying, "The Good Lord only knows where that document, is," 
he should be able to say, “FUREKA!" 


SUMMARY 

“A Must: Retter Information Retrieval" is the title of an article appearing in 
the January 11, 1960, issue of CHEMICAL FNGINFERING, The article begins, "If engineers 
are to become more efficient, one need is to be able to find what's gone before. 
Mechanized searching's now a must. In the world of engineering, the price of ignorance 
is technological surprise. With the acceleration of engineering development in the 
‘60's, chemical engineers will become more and more concerned with this threat. 
Engineers must exercise particular care to avoid these traps: 

Not knowing what one ought to know. 

Not being able to recall what one once knew, 

Starting a project already completed elsewhere. 

Repeating work already in the public domain, 

Knowing less about a given subject than the organized intelligence network of 

your competitor.” 


The problem is further complicated because a large and significant part of the 
world's scientific literature is published in foreign languages, with which so few of 
our American scientists are familiar. Currently, more technical articles are being 
published in Russian than in German; more in Japanese than in French; and more in 
Chinese than in Italian. Computer translation of these foreign publications is one 
facet of Information Storage and Retrieval currently under development. Russia has a 
huge task force of highly trained scientists working in its information retrieval cen- 
ter (known as VINITI), @ large part of which is devoted to translating and disseminating 
to every part of the Soviet Union the scientific knowledge contained in technical 
publications from all over the world, 


Various undertakings, both government and private, are well along in this country, 
for the solution of the information problem. 


The first step in the establishment of an Information Storage and Retrieval Sys- 
tem is to develop the philosophy and indexing system which will most efficiently and 
economically serve the needs of the particular organization. If the document collec- 
tion anticipated is relatively small ard homogeneous, a simple index and retrieval 
system will suffice. If the collection is to be extensive and hat heterog Ss, 
with considerable depth and complexity of indexing required for effective retrieval, 
then it is advisable to use a computer for storage and retrieval of information. But 
the system should be well thought out and developed first, before putting it on a com- 
puter. Otherwise, expensive experimentation may cause one to declare the computer, 
rather than the system, ineffectual. The developmental stage may very well require one 
to two and a half years, depending upon the size and the capability of the staff 
assigned to the task, 
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DO YOU MAKE THESE Q.C, MISTAKES IN BUYER 
AND SELLER RELATIONS? 


Dorian Shainin 
Rath & Strong, Inc. 
Management Consultants 
Boston, Mass. 


"Our inspection department just rejected the entire shipment of space pumps 
we worked so hard to get in from Satellite Accessories, Inc., this week!" 


The Purchasing Manager's reaction to this announcement went from an ex- 
pression of utter disbelief, through despair, and on to mounting anger. If 
you were to ask Mr. Beyer, ‘Who made the Q.C. mistake involved in the 
assembly floor crisis this rejection will precipitate?" his first words might 
touch upon thoughts like, "If you look hard enough you can find a reason for 
rejecting anything that was ever built - and our inspectors don't seem to 
realize we are supposed to be in business to make a profit," 


Of course, if instead you were talking to Mr, Cue, the Q.C,. Manager, he might 
answer the question by explaining, ''We have a vendor-rating plan, which has 
been in effect for some time now, and I wonder how much attention Purchasing 
has paid to the rating for Satellite Accessories, Inc. Why, they act as if they 
are surprised at this rejection! That supplier's rating is borderline." 


What would a trip to the vendor's plant produce as an answer to the question? 
We were surprised; they didn't berate receiving inspection of the Count Down 
Repeater Co. They described a complicated engineering change procedure 

this customer had: a few hundred changes per week, the disposition of material 
in process was not always clearly stated, ''We don't know about this case yet, 
Sut probably Engineering, Inspection, Production Control and Purchasing there 
do not really agree yet whether these particular pumps should be reworked, 
scrapped, or accepted as is, They were made to the same drawings as the last 
shipment, which was accepted. Maybe we'll get a change notice tomorrow that 
will tell us to do something different with them, and we'll have to reply that 
they've already been shipped. " 


"Actually", the Satellite spokesman, Mr. George Maker, went on, "they need 
the pumps quite desperately. They will probably find a way to use them. Last 
month we had separate visits from representatives of their Purchasing, Quality 
Control, and Engineering Departments, Their interests were different, and 

it would be impossible for us simultaneously to do what each group wanted. So 
we give first priority to what Purchasing wants, Frankly, we wouldn't be 
sorry if they took their business elsewhere. We do have customers with whom 
it is much easier to work." 


We thanked him for his revealing explanation, 
Who made the Q,C, mistake? Everyone we heard from can't be right. 


Actually no one was correct, That's what makes carrying product through its 
cycle from concept to delivery to ultimate customer, at times, so frustrating. 
You probably don't have to go far back in your memory to list a number of other 
Q.C, situations that were mixed up in some way with buyer and seller relations. 


"We found the rest of that lot you rejected from that sample to be less defective 
than the AQL we agreed upon with you," one vendor patiently explains. 


"It wasn't like that when it left our plant." 
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"You must have a new inspector on that job." 
"Your other plant accepted a shipment from this same lot!" 


"But you didn't say that in your specifications, We figured our costs on the 
stated requirements." 


"Other customers are finding our product satisfactory, " 
You can imagine the customer's inspection representative's replies. 


And while this list grows, we hear Purchasing earnestly explaining, ''No one 
else will agree to make it for us, "' 


Or asking, 


"Do you realize what kind of a delivery date we would get if we had those re- 
quirements ?" 


The questions, accusations, and words of advice multiply as we get further into 
specific problems of quality of design, and of quality of conformance of parts 
to the design requirements, 


Are companies really making Q.C. mistakes? If situations like these are not 
uncommon in week-to-week activities - yes. 


When quality is really controlled, problems such as these approach the vanishing 
point. This means, of course, Q.C, has to go beyond control charts and 
sampling plans - far beyond records that tell or imply to someone else that they 
are not doing as well as they should, 


It is, of course, impossible to become specific without restricting ourselves to 
a given plant - whether it be a seller or a buyer - in a given industry, We have 
felt this paper could perhaps be more useful in describing fundamentals. If 
they appeal to you, it will be your task to adapt them to fit your problems. 


The first six fundamentals we call defect-prevention ones; and the last five, 
defect-correction fundamentals. 


1) The buyer and seller should exchange outlines of standard practice. Bring 
out into the open, early in the relationship, the differences that may exist. 
Reaching the necessary agreements at that time will be considerably less costly 
in time, tempers, and treasury department matters. 


2) Tolerances should be considered as absolute, By this statement we mean 
that all deviations should be submitted to the equivalent of Material Review 
Board action; we mean consideration of the acceptance of non-serious devia- 
tions provided they are non-repetitive, or provided suitable corrective action 
had been initiated. We also recommend against the practice of reaching agree - 
ments between buyer and seller as to Acceptable Quality Levels for specific 
groups of characteristics. Such practice is in effect saying, ''The engineer who 
set the limits has been somewhat conservative, in view of his limited knowledge 
as to just how far he can go without getting into some difficulty, '"' In most cases 
that statement is quite true, But the practice in effect goes further, "'--so, we 
shall have the Quality Control Department (or some buyer-seller group) seta 
percent defective of a distribution that can extend beyond the tolerance limits, 
and the sampling plan will be selected that has a high probability of acceptance 
of this percent defective. '' That, unfortunately, is setting new, broader limits 
yet with the same limited knowledge as to just how far one can go without 
getting into some difficulty. 


Under the eleventh fundamental, "Realistic Tolerances Can Be Determined, " 
we shall describe how one can find how far he can go. 


3) Get production started on the right foot. If five units, parts, or what have 
you, are selected at random from among the first fifteen to twenty produced, 
they can be extremely useful in helping to comply with this fundamental, 
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A tag on each unit should record three readings of each important characteristic 
as measured on the seller's equipment, and some lack of repeatability must be 
evident! If not, the equipment is not sensitive enough to show its own vari- 
ability. Sometimes that situation obscures important product variability. 


The range among the five units, when converted to standard deviation by the 
d2 factor, should show at least eight standard deviations for the tolerance as 

a rough but not-to-be-neglected check on process capability.(1) The extra 
two standard deviations beyond the conventional six allow for some of the 
sampling error and for unavoidable shifts of process average coming from new 
raw material, replaced tools, and change of operator. 


Three more sets of entries should go on the tags from the readings on the 
buyer's equipment, when such is used for acceptance purposes. Important im- 
provements in equipment repeatability and correlation may be indicated by the 
comparison of the two sets of entries. 


Then find out if all five units will fit, function, or whatever will be expected of 
them, It is not unusual to find that some important characteristic had not been 
specified in the original design, 


4) Classify characteristics functionally, not defects operationally, for acceptance 


sampling assurance levels, Far more agreement as to the classification of the 
same characteristics by several classifiers can be obtained, A characteristic 
is called MATING if its primary function is to contribute to a fit or an over-all 
circuit characteristic. It is INDEPENDENT if the assembly can fail to function 
when it alone is beyond limits. And it is UTILITY when it is neither mating 
nor independent, being primarily descriptive rather than functional, 


This fundamental recognizes the variation of opinion among people who are 
classifying defects as CRITICAL, MAJOR, or MINOR, on the basis of the 
seriousness of the possible consequence of the characteristic being defective. 


Independent characteristics get a tight sampling assurance level; mating a 
more liberal level on the basis that two or more will combine to give a com- 
bination characteristic, the risk for which should equal that of the independent 
one; and utility gets only a sufficiently large sample size to identify that the 
characteristic is not defective to a widespread extent in the lot. 


Conventionally critical characteristics are assigned 100 percent inspection, 
major a tight AQL, and minor a more liberal AQL. If one interprets this to 
mean, respectively, nothing, a little, and a moderate amount beyond tolerance 
is to be permitted, then the previous discussion under the second fundamental 
suggests this thought: the engineer can increase the tolerance a moderate 
amount on minors and a little on majors -- now they would all be "critical" 
from the viewpoint that nothing more should be allowed beyond limits. 


The real problem seems to be that the practice of classifying defects is an 
attempt to control quality through inspection, Inspection does not control 
quality effectively; quality is best built into the product by those who produce 
it, the manufacturing people. 


5) Control productfon to keep parts away from the region of the limits. 
Duplicate gages, instruments, and test rigs seldom give extremely close, 
similar results because of many factors, the worst of which is usually wear 
differences. Whenever parts are near limits, as when material is sorted or 
screened at a limit, the seller and the buyer can find themselves mixed up in 
some discussions not particularly conducive to good relations. Process 
quality control devices for complying with this fundamental are, in order of 
decreasing complexity, X and R control charts, narrow limit gaging plans, 
and the PRE-Control technique, 


(1) Naturally, as more data later becomes available, the statistical precision 
of this check can be improved. 


Se 
: 
3 
<a 


362 1961 ASQC CONVENTION TRANSACTIONS 


6) The buyer should inspect to a defect-prevention schedule. When the buyer 
inspects for acceptance outside the seller's plant, first priority should be 
given any sets of the five tagged parts described under fundamental number 
three. Next on priority should be the first full lot of any production run, 
followed on the list by those parts urgently needed by the buyer. Last, but 
also on the priority list should come any material that, say, had been received 
more than two weeks ago. It is unfortunate when a detectable error is not 
found soon enough so that more material is produced with the same error. 


The next five fundamentals are defect-correction ones: 


7) The buyer should furnish statistical evidence of the quality characteristics 
found by inspection. These, preferably, should be in the form of histograms 
of random samples. The seller can,do a better and quicker job of correcting 

a defective condition when he sees that the trouble is a narrow distribution 
way off center, or two distributions in different positions, or a distribution 
that is too wide. ‘ 

8) The buyer should return one example ot each rejection found, even though 
the parts may be usable with or without rework by the buyer. Trouble shoot- 
ing by the seller can be considerably facilitated when he has more than a 
description or a picture of the questionable part. Being able to handle the 
part, trying it in the manufacturing fixtures and inspection gages can cut short 
many weeks of supposition as to what could have caused the apparent difficulty. 
In several cases, the returned part made it immediately clear that the trouble 
was centered in the buyer's gages or interpretation of requirements rather than 
the seller's. But it took the return of the part to the seller to bring the fact 
out quickly and clearly. 


9) Personal contacts between the buyer's representatives through its Purchasing 
Department and the seller's Sales Department on quality questions are to be 
preferred, whenever possible, over telephone or mail contacts. We have wit- 
nessed weeks of misunderstanding cleared up by one short visit too many times 
not to include this point as one of the fundamentals for good relations. 


10) The seller should investigate troubles in his plant in such a way as not to 
bring about an inadvertant change in quality standards, This principle also 

has an application among the buyer's inspectors, Inspectors have a natura! 
tendency to tighten up, subconsciously, in their evaluation of product for about 
a week or two if recent operating troubles are mentioned to them, as may well 
be the case during a search for clues as to how something undesirable got by. 
Unnecessary rejections can be avoided often if this fundamental is kept in mind. 


11) Realistic tolerances can be determined, if the need is sufficiently urgent, 
to ease a buyer-seller problem. The technique is called experimentation by 
the Random Balance method, It involves checking out the separate effects of 
many variables simultaneously with about 30 test runs. Each variable is 
arranged at random in its levels with every other one, The analysis consists 
of a correlation plot of each tested characteristic against the output. Because 
of the random assignment of levels, the regression line in each case represents 
the estimate of the effect of each variable alone, all the others having been 
balanced out and going into the cause of scatter of the points about the regression 
line. The plot involving the most important variable will have the greatest 
shape and least scatter. A simple graphical procedure on each plot shows the 
extent of inputs that will guarantee outputs within their limits. 


We should like to go on. We have strong feelings about survey teams, source 
inspection, and vendot-rating plans - all contributing to Q.C. mistakes in 
buyer and seller relations. Understandably, the subject is a big one - never 
completely finished no matter how well you approach it in your business, But 

it might pay to start to go after those fundamentals which have a major influence 
on these relations. 
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A CONTRACTOR WRITES A MILITARY SPECIFICATION 


Herbert B. Epstein 
Vought Electronics Division 
Chance Vought Corporation 
Dallas, Texas 


BACKGROUND 


In the late 1950's the Special Projects Office of the Bureau of Ordnance, United 
States Navy was established. Its urgent mission was to develop an operational Fleet 
Ballistic Missile Weapon System by 1960. ‘This Fleet Ballistic Missile was to have the 
capability of being launched as a long-range missile with a thermonuclear warhead from 
submerged nuclear-powered submarines. It was to be the United States' dramatic answer 
to the Soviet challenge in the complicated techni-political space age. 


The tightly-compressed time schedule for accomplishing this fantastic goal in- 
corporated a program for Quality Assurance and Reliability to insure that the vehicle 
and a vast complex of support equipment would be operational at the time it was needed. 
The Missile Evaluation Branch of Navy BuOrd's Naval Ordnance Laboratory, Corona, Cali- 
fornia was given the responsibility for developing with the contractors a quality and 
reliability system that would assure needed dependability during fleet operation. 


OBJECTIVES 
The planners of NOL, Corona, determined that several basic ground rules must apply. 


(1) Since there was such diversity of responsibility, it was necessary to 
establish minimum contractor quality and reliability system standards. 


(2) As a means of determining progress made by contractors in fulfilling the 
quality and reliability responsibilities, a system for determining progress 
against these standards was required. 


(3) Since various government inspection agencies would have cognizance over the 
operation of these different contractors and since training of government 
inspection and quality personnel varied from agency to agency, it was neces- 
sary to establish a guide for these agencies to utilize in implementing the 
quality and reliability program. It was also necessary to have a means for 
reviewing the progress made by these agencies in implementing and exercising 
surveillance over the contractors' programs. 


The magnitude of the job presented by these basic ground rules and the need to 
develop such specifications within a very short time period required the immediate 
application of experienced quality and reliability personnel to this program. Further, 
since these requirements were to be incorporated in the contracts of the FBMWS progran, 
it was necessary to have these documents prepared with the contractors' viewpoint taken 
into consideration. 


SELECTION OF CONTRACTOR 


To accomplish the project objectives and to meet the timetable involved, it was 
necessary for NOL, Corona to select for this task an experienced company which could 
make the proper type of talent available for the length of time needed. Chance Vought 
was selected as the contractor to prepare the specifications, which was titled: "De- 
velopment and Documentation of Quality Control and Reliability Standards for the Fleet 
Ballistic Missile Program". 


At Chance Vought, the Quality Control Department assigned the responsibility for 
successful completion of this program to the Chief of Quality and Reliability Develop- 
ment. It was his responsibility to recruit the key Quality Control and Engineering 
personnel required for the project and to direct the Chance Vought effort. 

DEFINITION OF TASK 
Management of this effort boiled down essentially to three key elements: 
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(1) Clearly defined objectives and policies. 
(2) Rapid and regular communication and co-ordination with NOL, Corona. 


(3) Assignment of a fulltime, experienced quality-reliability team to prepare 
the specification. 


TEAM OPERATING POLICIES 
Team operating policies were established as follows: 


(1) A special task group from Quality Control and Engineering will be assigned 
fulltime to the development and preparation of the subject standards and 
rating systems under the direction of the Chief of Quelity and Reliability 
Development. 


(2) The standards and rating systems developed should be acceptable to Chance 
Vought, if applied, as a test of the business soundness of the standards. 


(3) The continuous reliability and quality control concept in use at Chance 
Vought and expressed in the Department of Defense document developed by 
the Ad Hoc Committee for Guided Missile Reliability shall be the basis 
for the standards. 


(4) The standards and the rating system should be developed on a functional 
ih than a phase basis (i.e., pre-prototype, prototype, production, 
etc.). 


(5) All key sections of the standards shall include the "Concept and Approach" 
applicable to the subject. 


(6) The concept of "feedback loops" shall be prominent in all phases of the 
document, with particular emphasis on contractor - subcontractor relation- 
ships. 


(7) The standards will determine policies and end results of a contractor 
system rather than dictating “how to do". 


(8) The rating systems will be developed to highlight weaknesses as a guide to 
correction rather than for "rejecting" contractors. 


TEAM OPERATION 


Since this was a handpicked team, experienced in the Quality Control and Relia- 
bility fields both with aircraft, (F7U-3, F8U-1 and -2) and with the Regulus I and II 
guided missiles, each member was assigned responsibilities, according to his experience, 
to develop basic outlines for the various sections of the documents. Included in this 
breakdown were sections on "General Management", “Design Control", "Procurement Control" 
"Production Control", "Data Reporting and Corrective Action". It was recognized early 
in the development of these standards that MIL-Q-9858, NAVORD instruction 4355.22 and 
MIL-G-14461 (ORD) all described general Quality Control requirements applicable to 
military contractors but it had been determined by BUORD that, for the Fleet Ballistic 
Missile Weapon System, each of these documents, although generally applicable, was not 
sufficiently specific. The team, however, was required to be completely familiar with 
the key specifications associated with the project. 


DOCUMENT DISCUSSION 


The key document developed was MIL-Q-21549A (NOrd) Military Specification, "Quality 
Assurance Program Requirements for Fleet Ballistic Missile Weapon System Contractors". 
This was the very heart of the Quality Assurance Program for the Fleet Ballistic Missile 
System. Its intent was to define the requirements for the contractors and to be in- 
cluded as part of the contract for each of the programs. 


A review of the document will show that major emphasis was applied to design 
control and procurement control as well as to the traditional quality control effort 
associated with production. One of the most vital areas of emphasis was concerned with 
test equipment design and fabrication. The reason for this was that the combined experi 
ence of Chance Vought and NOL, Corona had indicated that many of the difficulties of 
prior programs had centered around lack of compatibility of the test equipment to the 
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product and poor co-ordination of test equipment used by prime contractors and subcon- 
tractors. 


The result was a dewanding document for a very demanding program. 


The second document, entitled "Guide for Evaluation of Contractor Quality Assur- 
ance Program", was intended for the use of NOL, Corona in evaluating contractors as- 
sociated with the program. It was recognized that in any contractor evaluation the 
contractor and the cognizant government inspection agency had to be part of the team. 
The purpose of the survey was for corrective action purposes. 


The third document, entitled “Guide for Government Inspection Agency", was pre- 
pared as a co-ordinating document between the various government services supporting 
this specific program. Its purpose was to establish basic ground rules for the in- 
spection agencies in harmony with the overall quality assurance program and to high- 
light those areas that were considered essential for assuring the success of the 
program. It recognized, however, that the management of each of these inspection agen- 
cies differed and that the contractors being serviced varied both in the scope and the 
type of product involved. But it did provide a guide by which the agencies working 
with NOL, Corona could better direct their efforts to the FBMWS program in the minimum 
time scale involved. 


The fourth document, entitled "Guide for Review of Government Inspection Agency", 
was developed for joint use by NOL, Corona and the specific government inspection agen- 
cies. Its purpose was to highlight potential trouble areas and to provide a basis for 
improvements, recognizing the besic autonomy of the inspection agency. 


GENERAL OBSERVATIONS 


During the preparation of these documents, both contractor and government view- 
points were expressed and serious debates with regard to emphasis as well as to require- 
ments were resolved. In some cases the contractors' viewpoints prevailed and in others 
the experience of the government as a basic surveillance agency prevailed. The vast 
majority of the requirements, however, were mutually agreed upon very early in the 
program. 


Fvents subsequent to the publication of these documents leads one to draw some 
conclusions. 


(1) A contractors' viewpoint did permit the publication of requirements that 
had been tested in industry. 


(2) The FBMWS Quality Assurance program was significantly accelerated by this 
effort. 


(3) ‘Two groups, one government, one industry, had their viewpoints enlarged by 
this effort --- to the benefit of both. 


(4) Any contribution that this effort makes, however modest, improves the de- 
terrent capability of our country at a time when it is needed most. This 
is the real payoff. 


is 
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MCTSA SEEKS QUANTITY WITH CONFIDENCE , 


Rene L. LaBonte, Assistant Chief, Technical Division 
Military Clothing and Textile Supply Agency, Philadelphia, Pa. 


Generally speaking, you have to give something to get something. The Military 
Clothing and Textile Supply Agency accepts this quid pro quo. The Agency is seeking 
the installation of quality control by all contractors or potential contractors; it 
guarantees that an acceptable quality control system will require less red tape for 
the acceptance and shipment of product. Before discussingthe quality control program, 
the mission of the Agency must be placed in proper perspective. 


MISSLON 

First, the Agency is an outgrowth of the Department of Defense Single Manager 
concept for specific commodity fields. It was established to eliminate existence of 
duplication and overlapping in procurement, storage and distribution, as well as other 
phases of supply management. For clothing and textiles, the Secretary of Defense 
designated the Secretary of the Army, then Mr. Brucker, as the Single Manager. Under 
him, through the Army chain of command, Major General Webster Anderson was appointed 
Executive Director of MCTSA in July 1956. The Agency integrated the four separate 
supply systems of the Army, Navy, Air Force, and Marine Corps and became a unified 
supply agency with full responsibility to meet the wholesale requirements of the four 
Services. A separate paper could be written about its success and overall achieve- 
ments. For example, millions of dollars have been saved by reducing the number of 
storage and distribution locations, use of an electronic receiving network, use of 
an automatic data processing system for a sensitive inventory control, the standardi- 
zation of items, the modernization of procurement methods, and others. 


Second, the scope of items assigned to the Agency includes more than is denoted 
by the words "clothing and textiles." This specific commodity field encompasses 
clothing in the sense of coats, trousers, drawers, and the like; textiles such as 
yard goods, undershirts and hosiery; and many related items such as various types of 
footwear and boots, tents, body armor, mattresses, field packs, embroidered and metal 
insignia, belts, flags, tapes, webbing, helmets. The Agency is responsible for approx- 
imately 5,000 different generic* items. Dollar-wise, the title of an informative 
address by General Anderson is explicit, namely, "A Two Billion Dollar Quality 

(1) 
Operation." The vastness of its supply system is evident. What may not be so 
evident, however, is the tremendous, continuing, and challenging responsibility to 
maintain a supply readiness keyed to peacetime needs and economy and instantly con- 
vertible to emergency conditions. 


Third, the Agency is comparable to any large industrial firm in having specific 
responsibilities for prudent management, efficient operations, and the maintenance 
of an economical inventory. It is equally concerned with satisfying its customers 
(the four Services). Being a military organization, it has the added responsibility 
for military preparedness. Current world conditions indicate the ever-present powder 
keg which could explode into a brush fire, guerrilla or limited war. Therefore, the 
Agency must be geared to supply large quantities of product with confidence to such 
areas directly and rapidly. Toward this end, its active and potential contractors 
must be depended upon to produce quality product. 


GENEALOGY 
The basic principles of quality control by the Government were developed in an 
excellent article entitled "Quality Control Management in the Department of Defense," 
written by John J. Riordan, Office of the Assistant Secretary of Defense, Supply and 
2) 


Logistics. The Department of Defense promulgated a series of broad policies to 
encompass total quality control. The Army, Navy, and Air Force had to translate the 


* The term "generic" refers to a kind of item. For example: Shoe, Men's, Service, 
Dress, Black or Shoe, Men's, Service, Dress, Brown. Each size of the item is 
assigned a federal stock number; therefore, each size is a line item. 
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(3) 
policies into practice. An Army Regulation stated that when there is satisfactory 
evidence of good quality, which is the result of inspection and/or quality control by 
the contractor, Army inspection will be adjusted to a minimum consistent with assurance 
that supplies conform to contractual requirements. 


OBJECTIVE 

Major General Anderson set his sights and charted a course to obtain his QC 
(quantity with confidence) through industry QC (quality control). He launched an 
intensive program to promote product quality control, with emphasis on statistical 
quality control techniques by Agency contractors, as well as by the soft-goods indus- 
tries as a whole. Farsightedly, the success of this program envisaged better product, 
increased industry capability, and timely delivery. What follows summarizes the 
progress made, the status quo, and the projected plans, all of which, it is hoped, 
will be sufficiently informative to stimulate and increase industry interest in 
becoming Agency contractors. 


ACHIEVEMENTS 

A procurement innovation directly associated with the objective of product quality 
control is worthy of first mention, namely, the Qualified Manufacturers List for 
clothing items. This list was developed to eliminate marginal operators and irrespon- 
sible bidders and, conversely, encourage and increase capable and responsible bidders. 
The QML requires that before a firm may be placed on any bidders list, it complete a 
detailed application and questionnaire which demonstrate satisfactory manufacturing 
capability, technical know-how, quality of production, business and financial integ- 
rity, and adequate plant facilities and trained labor. Applications are then reviewed 
by a board which is assisted by a staff of legal and technical specialists. Plant 
wisits are made to verify information furnished by applicants. Only firms placed on 
the QML may submit bids. 


The quality control program for the soft-goods industries had to be evolutionary 
because it was revolutionary. Hence, the Agency planned to execute the overall plan 
in three successive phases, as follows: First, contractor testing of materials or 
-omp ts; d, contractor examination of product; third, contractor control of 
production processes. It was foreseen that the successful execution of the three would 
result in the ultimate objective, viz., a quality control system in effect by each 
Agency contractor. The first two phases have been fully implemented; the final phase 
is underway. 


Contractor Testing 

Agency contracts require that contractors test, or have tested, materials or 
component parts to be used in the manufacture of the end item, as well as the end item 
when testing is applicable. The contracts provide that, at the time of presentation 
of a lot for government acceptance, contractors also present certified laboratory test 
reports on all tests required by component and end item specifications. These test 
reports are examined in detail by the Quality Control Representative (formerly called 
Inspector) to whom the lot is presented. If the reports show that all test require- 
ments have been met, they are taken as direct evidence of component acceptability. 
The interests ef the Government are protected through the medium of verification 
testing. Occasionally, with a frequency based on quality history, the Quality Control 
Representative draws a sample of the product tested and sends it, accompanied by the 
contractor-furnished test results, to the MCTSA laboratory in Philadelphia. The same 
tests are performed on the material and the results are compared and evaluated, in 
detail, with those furnished by the contractor. Correlation is the objective so that 
acceptance can continue to be made on plant test results. 


The Agency developed an attendant program to promote more process control by 
component manufacturers. The program is called Acceptable Suppliers List. In brief, 
it provides that if the supplier of a component part establishes and maintains an 
acceptable quality control system, acceptance will be made on the receipt of an ASL 
certificate in lieu of test results. The Agency protects its interest by occasional 
verification testing, examination, and plant surveys. The ASL was commenced for 
certain designated components. Currently, it is applicable to the majority of compo- 
nents procured by MCTSA contractors. Hopeful that this paper may be used as an 
immediate or future reference, the components on the list are: artificial leather, 
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shoulder pads, sleeve heads, coatfronts; labels, woven, printed; snap fasteners, 
insignia buttons, metal hardware; webbing and tape, braid, woven lace, draw cords, 
draw strings, elastic webbing, elastic tape and cord, ribbon, hanger loops; plastic 
and pearl buttons, plastic buckles; plywood, for other than furniture; silesia, 
buckram, cheesecloth; rope, string, twine; sewing thread, yarn, gimp; slide fasteners; 
small leather components; barrier material, moisture and vapor transmission; burlap 
and burlap tubing; naphthalene; fiberboard, paperboard, packaging paper and wooden 
shipping containers; steel strapping; polyethylene bags; bottom fillers; counters, 
outsoles, insoles, midsoles, welting, heel pads; nails and tacks, shoe last hardware, 
fiber counters, shanks, eyelets; counters (plastic); heels, soles, bases; box toes, 
felt tongue lining, shoe laces, insole reinforcing tape (gem duck), vamp lining, vamp 
and quarter doubler. For an up-to-date listing of items associated with this program, 
interested firms should direct their queries to the Technical Division, MCTSA. 


Contractor Examination 


Agency contracts require contractors to perform visual examinations set forth 
in the contracts and/or specification quality assurance provisions before presenting 
lots for government acceptance. Such provisions stipulate the standards against which 
to inspect such as the classification of defects, acceptable quality levels, and the 
use of statistical sampling plans. Government acceptability of the contractor's 
examination is determined by means of verification examinations. The contractor is 
required to score on an examination record all defects found in the sample examined 
for each lot. At the beginning of a contract, the Quality Control Representative 
performs his verification by examining a sample of the product from the lot offered 
by the contractor as being acceptable. If his findings are within the acceptable 
quality levels specified in the contract, and are statistically comparable to those 
of the contractor, the lot is accepted. After a predetermined number of lots are 
found to be acceptable and comparable, acceptance is then made on contractor results 
without verification. Thereafter, the Quality Control Representative performs his 
verification examination only on a skip-lot basis, providing the reliability of 
contractor results is maintained. This procedure enables a contractor to ship his 
product at times more convenient to him and to be reimbursed more quickly for product 
shipment. 


ULTIMATE GOAL 
CONTRACTOR QUALITY CONTROL SYSTEM 
It is worthwhile repeating that the two phases, contractor testing and 
contractor examination, are now de facto accomplishments; that this third phase is 
only the early stage of implementation. Contrary to expectation, Agency contractors 
did not voluntarily install the third important measure, namely, production process 
control. As a consequence, Agency Quality Control Representatives have had to 
accomplish too many lot-by-lot verifications because of statistical noncomparability 
and, in some cases, because of outright rejection of lots presented by contractors 
as being acceptable. Nevertheless, the objective of accepting with reliability is 
within reach. Since the fall of last year, the Agency developed a Quality Control 
System Requirements clause and placed it into Invitations for Bids for several items 
as pilot runs. In essence, the requirements provide that a contractor will design, 
document, and establish his own quality control system subject to the approval of the 
Agency QCR. When a satisfactory system is in effect, the Agency QCR's visit to the 
plant is not only infrequent but, for the most part, confined to an appraisal of the 
system including a review of the objective quality evidence. It is now within the 
realm of reality that quality control systems by contractors will be as automatic as 
contractor testing and examination have become. 


INDUSTRY ENDORSEMENTS OF QUALITY CONTROL SYSTEM 
Below are excerpts from letters sent to the Agency attesting to benefits 
realized: 


"...The advantages gained thereby are most gratifying. In the 
final examination alone, there has been a noticeable decrease in 
the number of defects found. As a result of this, I have been 
able to cut down my examination force considerably. Through 
the process inspection stations in the production line, we have 
eliminated almost completely any stoppages in production because 
of defective parts. Despite the necessary adjustment with the 
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installation of any new system, there has been no interference 
with production. On the contrary, the production foremen have 
found it to be of valuable assistance to them ..." 


",..We are well pleased with the workings of this plan and 
I am sure it is improving our quality on garments ..." 


",..It has taken us quite a while to digest the meaning of 
this new plan of production control. I will readily admit 
that it is quite helpful to us in making possible a better 
quality of production and we are quite happy and proud of 
attaining this status ..." 


",..The five most frequently cited advantages were: Reduction 
in cost, saves overall production time, less scrap, less repair 
and reworked material, and higher quality level of product. 
Other advantages are: elimination of 100% inspection, smaller 
production of seconds, free workers for other jobs, less 
machinery repair, more production per worker, increase in pay 
to workers, quicker acceptance by Q.M. inspector, defective 
raw materials spotted before put into production, increase in 
sales, uncovered better production methods, gained better 
knowledge of workers' capabilities, and greater customer 
confidence." 


PROMOTIONAL EFFORTS 

The above resume of the entire program belies the immense amount of effort 
expended, the trials, the tribulations, etc, The Military Clothing and Textile Supply 
Agency has received praiseworthy recognition for fostering quality control in the 
soft-goods industries, which were virtually in the dark concerning the application of 
this science. [It authored such pamphlets and manuals as "This Is Contractor Testing," 
“This Is Coutractor Examination," "Blueprint for Quality Control," "Contractor 
Inspection," "Evaluation of a Quality Control System." It conducted seminars, confer- 
ences and meetings, and prepared newspaper and magazine articles. 


TRAINING 

The wherewithal to accomplish so much was a challenge in itself. From the very 
beginning, a major training program in statistical quality control was launched. 
Military Clothing and Textile Field Inspection Offices were redesignated as Military 
Clothing and Textile Quality Control Offices. Personnel who met the necessary qualifi- 
cations were converted from end-item inspectors to quality control personnel. The 
average age of field personnel has been reduced from an average age of 55 years to 
approximately 42 years. The institution of a career development program resulted in 
increasing the staff with college-caliber personnel. To date approximately 300 Agency 
procurement inspection personnel have been trained in statistical quality control at 
the Quartermaster School, Fort Lee, Virginia. Key civilian personnel and military 
officers have received advanced quality control training at the Air Force Institute of 
Technology, Wright-Patterson Air Force Base, Ohio; at the Ordnance School, Rock Island, 
Illinois; and at selected civilian universities. The Agency played an important pert 
in the development of an advanced quality control course now also being given at the 
Quartermaster School, Fort Lee, Virginia. All of this training resulted in the 
promotional effort of the Agency to bring quality control to the attention of the 
industries with which it does business. Over 150 quality control plans have been 
developed by Agency QCRs, tailored to individual contractor plants. Recently, this 
Agency collaborated with the Philadelphia Section of the American Society for Quality 
Control and developed a basic course in quality control which is being given at the 
Philadelphia Quartermaster Center. The course was designed especially for manufac- 
turers in the clothing field. There are no educational prerequisites for the course. 
Its principal objective is to teach a basic understanding of quality control princi- 
ples and practices, 


OPTIMA- SEE 
Admittedly, this is a play on words. Whether unique or not, it expresses the 
Agency conviction that contractors will "see the best" in what a quality control 
system will do for them. In retrospect, it took years before the use of a scientific 
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sampling plan in the inspection of product was accepted as a valid, ingenious, and 
economical method to obtain a representative picture of a mass or lot of product. 

The same could be said about the acceptance of automation, now increasing at a rapid 
rate. Unquestionably, quality control is here to stay; the threshold has been reached. 
Only the final spurt remains. To this end the Agency will continue to foster and 
assist industry associations, as well as individual contractors, in the establishment 
of a "QC" (quality control) system so that it can realize its "QC" (quantity with 
confidence). 
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SIGNS OF OUR TIME WHICH FORECAST QUALITY 


Colonel Richard Stolle, Chief, Quality Control 
Headquarters Air Materiel Command, Wright-Patterson AFB, Ohio 


Quality Control today has been caught up in an era of rapid, we might say, explo- 
sive, change. One American philospher has noted that, "It took man 475,000 years to 
arrive at the agricultural revolution - another 25,000 to come to the industrial revo- 
lution. We have arrived at the Space Age in 150 years - and while we do not know 
where we go from here, we can be sure that we will go there fast." The evidences of 
change - whether it be time compression in the development, production, test, and oper- 
ational phase of weapon systems; a transition of buying complex systeme in small quan- 
tities; or increased subcontracting by the prime weapon systems' contractor - are in 
evidence all about us. 


let's look at some of the changes in research, development and production that 
have affected our way of life. It is well know history that a full scale mobiliza— 
tion was started during World War II. At this point in time, the Air Force as well 
as industry were aware that the existing methods of dealing with quality problems were 
inadequate. There were many signs indicating that the old inspection systems had been 
overtaken by the rush of technological advances in products, Industry and the Air 
Force had also come face to face with the fact that practical and economic considera- 
tions made it essential that duplication of inspection between the Governrsnt and con- 
tractors be eliminated. It was after this point in time, that the "Statistical Quality 
Control movement," gained its full momentum. Statistical quality control was developed 
to serve the needs of large production quantities and did a much better job than the 
old inspection systems, 


In the latter part of the 50s, however, we were in a state of transition. Where 
aircraft had been produced in hundreds they were now being produced in tens. Qualita- 
tive superiority was no longer being based upon the nation's ability to produce large 
quantities in which acceptance of a few defective items could be tolerated. Instead, 
the emphasis was on small quantities of very complex systems of aircraft and missiles 
requiring the utmost precision and quality in order that they would perform their 
mission the first and only time used, This trend will be greatly accelerated during 
the 60s. The Sputnik awakening also made the long term step-by-step process of develop- 
ment, prototyping, service testing, and getting into production impractical. National 
survival necessitated compression of the research, development and production cycles. 
This “time compression," as it was popularly called, imposed a terrific burden on 
everyone. This was particularly true of quality control, since definitive criteria 
for acceptance purposes was no longer readily available. Quality control found it 
increasingly important to keep a close liaison with engineering and production. 


Another significant sign of the times is the tremendous increase in subcontracting 
by the prime weapon systems contractor. Where the prime contractor previously had a 
relatively simple receiving operation to verify incoming material, the increased com 
plexity of subsystems defied any meaningful reinspection upon receipt. This, of course, 
considerably broadened the role of quality control. Industry as well as the Air Force 
found it necessary to station quality control personnel at all locations where contract 
services were being performed. There is considerable concern over the implementation 
of this system. Responsibility cannot be firmly placed and there are indications that 
with so many personnel involved each may be placing too much dependence upon the other. 


The increase in subcontract activity has resulted in a corresponding increase in 
requests for Government source inspection. This practice is not in accordance with 
AMC policy. The full impact of these requests for source inspection is difficult to 
ascertain, One integrating contractor, for example, has 8 major associate and 34 
supporting contractors, Each of these 42 contractors have Air Force prime contracts 
against which they issue a considerable number of subcontracts. The exact number of 
subcontracts issued by these associate contractors is not known but the integrating 
contractor alone buys material from 3,000 subcontractors involving some 27,000 items 
with the dollar value estimated at approximately 65% of the total dollar value of the 
contract. This integrating contractor is known to issue some 300 subcontracts each 
day and virtually all of these requests are approved for Government source inspection. 
The Air Force, as a result, is finding it difficult to cope with this increase in 
requests for source inspection. There are indications that some contractors may be 
relying on Government source inspection to provide assurance that material conforms 
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to the requirements of the contract. This, of course, constitutes non-compliance with 
contractual requirements for vendor selection and vendor control. In addition to 
MIL-Q-9858, there are several contractual documents, such as: MIL-R-26674, USAF Spec- 
ification Bulletin Nr 510, and AFBMC Exhibit 58-10, which contain specific requirements 
for vendor selection and vendor control systems. It should be noted that we in the 
Air Force do not perform source inspection for the benefit of the prime contractor, 

but act as an extension of the AFQCOR in the prime contractor's plant on material which 
is too complex to verify upon receipt at the prime plant. 


Second and third tier subcontracting has also become a way of life in present day 
methods of contracting and requires considerable emphasis in future procurements. The 
problem of producing and integrating the many specially designed components plus the 
fact that design experience is limited, is calling for a more active relationship 
among the various participants. It is well known that nothing helps to put things in 
perspective so much as a look at one's supplier's shop and talk with his people. 
Industry is finding visits to the vendors' plants increasingly valuable as it provides 
an excellent opportunity to pass along much needed information. It is particularly 
desirable that these second and third tier suppliers have a full appreciation of the 
part which their products will play in the over-all system and the technical problems 
involved in obtaining satisfactory final compatibility with other components or 
systems. In addition to these changes in our way of doing business, today's complex 
products have also had a significant impact upon the quality problems being experienced. 


The dependence of our latest weapon systems upon the complexities of electronics 
is great. Electronics, without question, has increased the capabilities of modern 
weapon systems because without it a workable missile would still be far in the future 
and Explorer I would still be in the province of science-fiction. These increased 
capabilities, however, have been made possible only at the cost of great increases in 
complexity. A ballistic missile, for example, contains from 10,000 to 300,000 separ- 
ate components depending upon the particular missile in mind and the degree of break- 
out made. The total lead time necessary for the development and production of missile 
and space systems today is determined - not as it was during World War II or the 
Korean conflict by airframe requirements - but by the time required to develop, pro- 
duce and integrate the maze of electronic gear. Electronics alone represents a greater 
investment in time, engineering effort and money than did the entire aircraft of World 
War II, For every airframe and propulsion problem there will be hundreds of electronic 
systems problems. This increased complexity, naturally, complicates the job of logis- 
tics support. 


The impact of unmanned systems is also particularly significant from a quality 
control standpoint. In the past, highly trained flight crews have been an integral 
part of the weapon system. During flight, this human element was in many cases able 
to adjust for slight variations in equipment or in some cases for outright equipment 
failure but the luxury of this compensating factor to correct for imperfections is no 
longer available. It is a grim fact that at the critical moment there may be only 
one chance to push the button. A second missile or an anti-missile-missile, in case 
the first one fails, may not be possible. The reliability and precision requirements 
of these weapons and the magnitude of the quality problems they present is a challenge 
to modern quality control systems. The need to integrate all activities which have a 
bearing upon quality is challenging the status of Quality Control Departments in many 
organizations. 


The most precious commodity available today is time. This resource must be con- 
served at all costs. In this environment the genesis of any modern quality control 
program must be in the design stage of the weapon, All too often this fact is ignored 
at the cost of redesign during the production or modification during use. It is 
important to note that many design deficiencies are not just "state-of-the-art" design 
problems but are common errors that could have been prevented by more effective control 
during the design and engineering stages. To alleviate these deficiencies, the effec- 
tive review of design by competent technical personnel is needed to weed out those 
obvious errors that so frequently arise. In addition, more emphasis must be placed 
on the design and specification of component tests to assure early detection of unsat- 
isfactory performance. Careful scrutiny of designs to assure that tolerances are 
realistic and permit economic production in the plant and maintainability in the field 
is becoming a must. The required performance cannot be acquired from a poor design 
by merely specifying unrealistic tolerances. Much can be accomplished in this area 
and study on improved methods and techniques is required. 


The importance of s metrology engineer, as a member of the design team is daily 
becoming more apparent. This technical capability on the design team would assure 
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that design tolerances are realistic and permit economic producticn in the plant as 
well as maintainability in the field. There are field calibration requirements at 

the present time which exceed the present “state-of-the-art.” Fortunately, past 
experience with similar problems has shown that many calibration requirements can be 
reduced both from the standpoint of accuracy and the number of individual calibrations. 
In the missile program, we find that a considerable portion of the maintenance workload 
involves calibration. This is not only because of exotic accuracies and the large 
number of calibrations; but a major factor in this workload is the inaccessibility of 
calibration points. An example of this is one of the missile support systems which 
has numerous pressure gauges so installed in the system that LOX clean lines must be 
cut, the gauges removed and calibrated, and then welded back into the system. It is 
quite obvious that much more consideration should be given to the accessibility of 
equipment for calibration in future design. 


As a result of the many difficulties being experienced in providing calibration 
support, Amendment 2 to Specification MII-D-9412C (USAF) has been developed which 
requires contractors to give adequate consideration to calibration requirements for 
the weapons/support system early in the development cycle. This identification must 
include the peculiar test equipment to support the weapon system, the common test 
equipment to support the peculiar test equipment; the standards required to support 
the common test equipments; and all the calibration data required on each piece of 
equipment listed. This identification will provide the Air Force an opportunity to 
evaluate and assure the calibration maintainability of the system during design. 
Past experience has shown that this procedure will pay large dividends if accomplished 
prior to production. 


Surprising as it may seem at this point in time, there are still some activities - 
both Air Force and contractor - who are reluctant to calibrate test equipment used for 
research and development. This is difficult to understand b the adequacy of 
design cannot be measured unless there is some certainty as to what is being measured. 
If the data generated are *%o be of any value, the accuracy of the test equipment must 
be assured, Instrumentation, in reality, is the intelligence gathering tool of re- 
search which separates fact from fiction. We have recommended to the Research and 
Development Command that they include the requirements of Air Force Specification 
Bulletin Nr 520 in all future R&D study contracts, 


In the production process there is a dire need for more gauges that measure, 
indicate, record, and control, The chemical industry is quite advanced in this re- 
spect but the machine tool industry has been slower to move in this direction. Maybe 
this is because the chemical industry is younger and has developed its instruments 
after the machine tool builders had stablized design. To effectively control quality 
during production, more and more of the tools must have automatic control features. 
This may require a look past many of the conventional methods of production measure— 
ment and search for news; maybe radically different types of measurement. The non- 
destructive testing field seems to be a very fertile area which can be applied to 
this problem. It is certainly a fertile area that deverves future research. 


A new link in the quality chain in recent years is the installation and check-out 
phase of the missile sites. The quality problems experienced at these sites are 
unique in that these operations present a completely different atmosphere, The same 
degree of control must be exercised in a heavy construction atmosphere as is exercised 
in the factory environment. This problem of maintaining control is enhanced by such 
factors as remote locations, widely dispersed areas, and high labor turnover. 


Many of the major problems stem from personnel as so often happens. The Air 
Force as well as contractors, is faced with difficult manning requirements for the 
next few years. Since these positions are of a temporary nature with high cost 
housing, and poor school facilities, it is difficult to recruit qualified personnel 
for site employment. In some instances, contractors have been able to offer pay 
inducements but in many cases they have had to resort to hiring inexperienced or 
partially qualified people from the local areas. This, of necessity, has resulted 
in less than an optimum operation. 


To add to this problem of exercising control in the installation and check-out 
phase, it was soon found that much of the quality control effort had to be dispersed 
to other areas, Many interface problems existed between site construction and instal- 
lation and check-out of the missile. Much time has to be expended in pre-acceptance 
inspection of facilities to get correction prior to acceptance of structures, From 
these experiences, it became apparent that the Corps of Engineers which by tradition 
was construction oriented needed to integrate their procedures with Air Force and 
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contractor quality control systems. Such problems as incomplete power conduits, 
improperly placed piping obviously will present problems during the check-out period, 
Another problem that hit with terrific impact was the control of cannibalization. 
Past experience at Vandenberg and Patrick Air Force Bases have shown that rigid con- 
trol pays dividends. 


We started by discussing some of the changes that have taken place in quality 
control and have reflected some of the challenges we face today. In the February 1961 
issue of “Industrial Quality Control," Dr Juran, quite aptly discussed some of the 
perils facing Quality Control Departments which continue to exhibit technique orien- 
tation rather than problem orientation. Along this line, let's briefly discuss the 
Air Force's decision to cease formal certification of processes, equipment and per- 
sonnel in such areas as heat-treating and plating. There were many signs that indi- 
cated the system had become technique oriented. It was found that these certifications 
were giving a false feeling of security to the purchaser as complete reliance was being 
placed upon the system. Users, were by contract, responsible to ascertain compliance 
to specification requirements; yet, Air Force certifications in many cases were being 
accepted at face value. These certifications, unfortunately, had assumed a routine 
day-to-day administrative atmosphere involving considerable paper shuffling and served 
little value from a quality control standpoint. There were indications that the rush 
of technological advances had overtaken the system and these processes needed to be 
placed on the same basis as other subcontractor supplies and services, 


In conclusion, there is an area of quality control that has been generally over- 
looked which could contribute much to the quality and reliability demanded today. It 
will be recalled, historically, that craftsmen existed not too far in the past. These 
individual craftsmen made the entire item and its perfection was a measure of his 
skill in which he took great pride. There followed in succession the period of fore- 
men quality control, inspection, and statistical quality control, aimed at controlling 
product fabrication in the plant. Gradually, more and more control was added to in- 
coming material. To round out the quality control program the preventative aspects 
were recognized and the design function was added to arrive at what is commonly termed 
total quality control. 


The many writings and discussions on total quality control have left out a very 
important function, which for lack of a better term, will be called operator quality 
control, Nothing is to be gained if we carefully control design, without making 
certain of the skill of the operator. Current problems attest the fact that a coor- 
dinated program of personnel quality control has often been lacking to a considerable 
degree. It has been found that quality control needs engineers in addition to its 
technical personnel and professional people to accomplish tomorrow's quality control 
job. Likewise, it might be equally profitable to have a personnel or training 
specialist aboard because company policies on hiring, transferring between jobs, 
promotions, and changes may not be conducive to a quality product. 


In the 60s the quality control loop must be closed. We have looked much too long 
for a system to substitute for the skill of the operator. The craftsman must be re- 
established as the Keystone of the quality control system. Quality control methodology 
and reliability techniques must be adjunct to the development of the operator - not a 
substitute for his accepted deficiencies, Our professional and engineering quality 
control personnel must be organized to support the operator in the design and develop— 
ment phase. We can best assist the production process by clarifying the intent of 
each requirement and including only realistic requirements in the product specifica- 
tions. 
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NAVY PROGRESS IN QUALITY CONTROL AND RELIABILITY IN 1960 


Vice Admiral George F. Beardsley, USN 
Chief of Naval Material 
Department of the Navy 


I appreciate this opportunity to talk to you about a subject of mutual interest 
and importance -- the progress being made by our country's Military-Industrial team 
in achieving a satisfactory and not too costly quality control program for the mili- 
tary's varied and multitudinous purchases. I hope to give you some of my own impres- 
sions as Manager and Administrator of the Navy's Material Inspection organization, 
rather than discussing technical problems which are adequately covered in the agenda 
of your Association's Annual Meeting. 


We believe the steps we have been taking in the past several years to improve 
our overall concept of quality control continue to show their basic worth. While I 
know all of you are thoroughly familiar with these steps, I would like to point out 
a few of the more important. The principal one is, of course, the continued high- 
lighting of the fixed and firm responsibility of our contractors to assure, prior to 
their submission for Government acceptance, that the supplies and materials being 
furnished conform to contractual requirements. Another major step is our decision 
that the objectives of a sound military Quality Control System are achieved more 
effectively by collaboration with, rather than by duplication of, similar activities 
in Industry. And, finally, I want to mention the dual goals of our own managerial 
efforts in the field of quality control. These are not only the prevention of defec- 
tive material from entering service, but also -- and of equal importance -- the obtain- 
ing of better material at lower prices. 


We need both better and cheap omp ts for our ships, airplanes and weapons 
if we are merely to keep up with the understandably insistent demands of the operating 
forces for increased capabilities in the tools of war with which they are equipped. 
Likewise, the rapid technological advances of our current world, which, of themselves, 
make these increased weapon capabilities possible, continue to result in greater com- 
plexity and cost. While I realize these are oft-heard statements used variously for 
many purposes, including excusing our own shortcomings, this state of affairs actually 
exists and is something with which we must live in increased harmony if the Armed 
Services are to carry out their basic missions. 


To do this, we must reiterate in ail appropriate forums the fact that improved 
quality assurance for all materials must continue to be a Joint Military-Industry 
responsibility necessitating cooperative efforts and attention at all management lev- 
els. On the whole, our suppliers have succeeded in the past year in finding effective 
measures for improving quality control systems. They have reappraised the standards 
and qualifications of personnel, established educational programs, reviewed quality 
control techniques in manufacturing and inspection methods, taken more immediate cor- 
rective actions resulting from feedback data obtained from the users of their equipment 
and materials, and, finally, have placed tighter control on materials purchased from 
subcontractors. The Navy in turn can -- and is anxious -- to help industry improve 
the overall Quality Control System. To do so, we are striving to improve the technical 
capabilities of our inspection personnel. Engineering staffs are being established in 
all General Inspection Offices. We have also increased the use of graduate engineers 
in our Quality Assurance activities. Personnel with appropriate academic background 
in the physical sciences are being recruited. In these ways, we are trying better to 
prepare our agencies to assist you in this important manufacturing function. 


Let me cite a few examples which have recently come to my attention which illus- 
trate our current quality assurance concept, the Military-Industry cooperation needed 
to attain maximum results, and certain recurring problems. 


As you know, the Navy purchases paint, enamels and lacquers for all Military 
services. In accordance with our basic principle, to which I have alluded, the paint 
manufacturer is required to maintain an effective and economicel quality control system. 
This covers receipt, handling, storage and testing of ingredient materials; methods 
and processing equipment; and procedures, controls and equipment for conducting tests. 
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Government acceptance of product is now based on the supplier's records of his own 
examinations and tests, with the Naval Inspector providing system evaluation, mon- 
itoring, and a minimum of actual production verification. The Government labora- 
tories, which formerly did routine acceptance tests, now are utilized only for random 
verification of the results of the supplier's own tests. This procedure gives us a 
dividend in that the Government laboratories are now freed to fulfill better their 
basic research and development missions. 


: Let us now turn to a different aspect. Here a manufacturer had no quality con- 

4 trol system to speak of, and was apparently opposed to installing one. As a result, 
this company, which was providing ordnance parts, was having extreme difficulty in 
controlling dimensions. In spite of several specification waivers that had been 
granted, rejections at his plant were exceeding 20%. User activities bitterly com- 
plained, partly because they did not "get the word" on the waivers, and partly 
because inspection at the manufacturer's plant had not reduced defects to an accept- 
able minimum, As a result of all this trouble, the supplier himself was losing 
money. More importantly, from our own point of view, Navy costs were increased by 
delayed material deliveries and by the necessity for detailed inspection of the 
supplier's material at our own plants in order to avoid working on defective material. 
The causes for all this trouble were finally located in the supplier's foundry which 
was producing non-uniform and often excessively hard castings. These were difficult 

; to machine within acceptable tolerances and also caused unpredictable tool wear. 

The manufacturer finally agreed to try more effective quality controls first in his 
foundry and then in his machine shop. Scrap production is now 2/3 lower; the sup- 
plier is making a profit, and is a satisfactory producer for the Navy. 


Now, let's consider a supplier who, while having a quality control procedure, 
had not taken the necessary initiative to make it work. This contractor was making 
a vital component for a-high priority project. Close inspection revealed errors in 
some elements. Poor manufacturing quality control was nullifying a generally good 
engineering job. Upon our request, the supplier instituted some corrective measures. 
Subsequently, the supplier's improved quality control system was evaluated but numer- 
ous procedures were still deficient. This time, the supplier was advised of specific j 
deficiencies in inspection methods; personnel and training; control of purchased 
material; use of feedback data; qualification for special processes; and non-conform- 
ing supplies. Continuing deficiencies of this type and evidence of poor product 
quality are still adversely affecting this supplier's relations with major contract- 
ing offices - Navy and primes. We have hopes, however, that further mutual coopera- 
tion between the contractor and ourselves will finally achieve satisfactory perform- 
ance, 


Let me touch briefly on reliability -- the basic purpose of quality control. 
Reliability requirements cannot remain static, but must be continually reviewed, 
based on operational experience. In addition, we must encourage and increase the 
interchange of reliability data among manufacturers and within the Military. I was 
happy to hear recently that the Services are effectively interchanging parts test 
data among ballistic missile manufacturers and the Services. This is excellent and 
should be expanded, Perhaps greater use of reliability incentives in contracts would 
be an added inducement to suppliers. I don't know, but I will have to look into 
this matter some more, 


As you know, we have increased our use of value engineering incentives in con- 
tracts, realizing the important contribution that design simplification can make to 
lower costs and to increase quality and reliability. We have many examples of this. 
An important one that comes to mind is the Navy's new amphibious assault ship which 
can launch helicopters and landing craft simultaneously. A new type of ballasting 
system, using gravity and air, has eliminated pumps and markedly simplified the pip- 
ing system. This design improvement saved one and one-half million dollars on three 
ships. The new system is faster, more easily controlled, simpler, and more reliable. 
Another example is an anti-submarine rocket launcher, where after taking a good look 
at the initial design, twenty-five per cent of the parts were eliminated from the 
control panels, The redesigned panel is less complex, more reliable, more functional, 
and easier to maintain and operate. In addition, costs were reduced seven hundred 
and fifty thousand dollars. I also heard recently of a seven dollar diode used to 
protect a transistor circuit in a component for the Polaris Submarines that has 
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been replaced by a more reliable resistor costing five cents. We have only scratched 
the surface in areas like these. Both Industry and the Armed Services must give 
greater attention to value analysis, especially during design. 


Gentlemen, I would like to leave one last thought with you. You are on the 
firing line, where production is taking place. You must daily see improvements we 
can make in our material inspection practices. Please let us have any suggestions 
you may care to offer. It is only through our joint efforts that we can become 
more efficient inspectors. If you have suggestions or complaints on how we admin- 
ister the Material Inspection Program, drop me a line - I have a willing ear and 
will recommend for adoption any reasonable suggestion which improves our products, 
lowers our expense of doing business, and increases the amount of quality material 
entering the Fleet -- material we badly need to accomplish our job. 
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QUALITY CONTROL-THE EVOLUTION OF AN OCCUPATION 


How the shift in the Government's quality control program changed the job 
requirements of Federal employees 


By William R. Collins, U. S. Civil Service Commission 
and 
Samuel S. J. Skolnik, Department of Defense 


The purpose of this paper is to report recent developments and trends 
in quality control in the Federal Government, the new skill requirements 
created thereby, and the formulation of personnel standards for positions 
affected by the resulting program changes. The positions affected are of 
a new kind, have a new purpose, and to some extent displace traditional 
inspection positions. 


In this age of increasing complexity many occupations are undergoing 
drastic changes. Therefore other program managers and personnel or other 
staff officials faced with a need for a major redirection of the skills of 
personnel to function properly in a situation transformed by a change in 
management environment may find a special interest in the facts presented. 
The general reader also may find that it provides a case history that is both 
timely and illustrative of the impact of modern technological advancements 
on our daily lives. 


INTRODUCTION 


Recognition of the need for the creation and development of sweeping 
changes in an occupation seldom occurs as an isolated action. On the con- 
trary, it msually represents a gradual recognition by many people that 
cumulative changes in policies and practices have significantly transformed 
the basic program thus creating a need to reevaluate personnel skill require- 
ments. So it evolved in the field of inspection in the Federal service. 
Personnel recruitment and retention problems became increasingly difficult. 
Ingenuity in making the existing personnel structure function satisfactorily 
in a changed environment was taxed to the utmost. Eventually the conclusion 
was reached that it was no longer possible merely to continue reorienting 
traditional types of inspectors to do the job. Management became aware of 
the need to do a major overhaul of its thinking about personnel needs for 
these jobs, rather than to continue trying to do additional patchwork to 
modify the existing structure. 


The new occupational standards for Federal positions in the Commodity 
Quality Control, Inspection, and Grading Group were developed by the Civil 
Service Commission with the cooperative efforts of the subject matter experts 
and personnel specialists in the agencies. These standards typify the 
beneficial results of timely recognition of the need to study jointly the 
effects of fundamental program changes upon the employee skills and knowledges 
required to perform new functions. 


EARLY DEVELOPMENTS IN QUALITY CONTROL 


Quality Gontrol encompasses all the techniques and procedures utilized 
during design, development, production, storage, maintenance and operational 
use to assure that quality is first built into a product and is then held at 


a suitable level for accomplishment of the product's intended use. The science 


of quality control is of very recent origin. Quality control principles 
developed during the 1920's were not accorded widespread application until 
World War: II created an urgent need for more efficient methods of production. 
Since then, quality control has attained nearly universal recognition both 
in Government and industry. It originated in the United States, but its 


applications are now worldwide. The rapidity of its emergence has been limited 


only by the availability of trained personnel capable of applying the new 
techniques. 
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NEW SKILL DEMANDS 


The application of quality control methods requires a combination of 
capacities and skills seldom previously demanded of a single individual. 
Quality control requires its practitioners to possess and utilize a combi- 
nation of technological, mathematical and administrative abilities all at 
an advanced level of accomplishment. To develop in the Federal service the 
type of staff required to apply quality control principles, it was necessary 
to create the proper balance of skills by recruitment from among the estab- 
lished career fields. In the early stage of quality control in the Govern- 
ment competent technologists, mathematicians, and administrators were 
brought into the program. In rare instances, individuals were discovered 
who possessed two, or even all three, of these qualifications. These 
individuals with a combination of diverse experience were helpful in providing 
a bridge of communication among other individuals with totally different and 
highly specialized orientation and background. 


Usually the new employees were introduced into organizations which 
formerly were staffed preponderantly with traditional inspectors. Frequently, 
these new employees were graduate chemists, physicists, engineers, food 
technologists, and the like, who had sufficient mathematical training and 
analytical ability to permit their ready conversion to quality control 
specialists. Most inspectors who had been employed prior to the advent of 
quality control evidenced trade or craft skills of a high order, but many 
did not possess extensive education or analytical capacity. However, many 
of these inspectors were trained to utilize simplified quality control 
methods under proper guidance, although they continued to perform inspection 
duties which primarily required trade or craft skills. 


QUALITY CONTROL PERSONNEL STANDARDS FROM THE SUBJECT MATTER POINT OF VIEW 


Prior to 1958 there were basic differences in viewpoint among the 
Federal agencies and among the various inspection organizations within the 
Department of Defense regarding the underlying principles and practices 
that constitute an efficient and effective inspection and quality control 
program. These diverse views were reflected in variations in the kinds of 
programs which existed in the inspection and quality control organizations. 
It is readily understandable that significantly different personnel require- 
ments were needed for the accomplishment of those extremely divergent pro- 
grams which ranged from those restricted to the simplest kind of visual 
product inspection to those incorporating the most highly developed and 
refined quality control techniques. However, since 1958 the Department of 
Defense promulgated a comprehensive set of policies which institute uniform 
quality control programs and practices throughout the military departments. 
To a considerable extent, similar policies and practices are now being devel- 
oped in the other Federal agencies. For example, quality control policies 
of the General Services Administration and the Department of Defense closely 
parallel each other. It is apparent that, with the establishment of a 
uniform set of basic principles and practices, agreement on the types of 
personnel required to carry out such programs could more readily be achieved. 


Thus, in conjunction with the establishment of a uniform quality 
control program, the Department of Defense determined that it was necessary 
and timely to review quality control personnel requirements and identify 
changes which were needed to accomplish its mission under the new program. 


The new Department of Defense policy placed responsibility on suppliers 
for performing required inspections and tests and for the development of 
systematic manufacturing and quality controls to assure satisfactory perfor- 
mance and reliability of equipment. The role of the Government under this 
policy was changed from determination of compliance primarily through direct 
examination of product to a one of verification that suppliers have estab- 
lished adequate manufacturing and quality controls and that inspections and 
tests have been properly accomplished. This verification of product quality 
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by the Government is achieved through a combination of direct product 
examination and review of suppliers' quality systems and records. As a 
result, Government and industry quality control programs now complement 
each other, rather than duplicate each other as in the past. 


The rapid technological advancement and ever-increasing complexity in 
weapons were additional reasons for the Department of Defense to institute 
a complete reexamination of its quality control personnel requirements. 
These developments forced a continual increase in the level of skills 
required for the evaluation and assurance of product quality. Pressures 
for the attainment of higher reliability levels in missions and other 
weapons systems resulted in the establishment of more refined and effective 
quality control techniques. Throughout Government and industry these 
developments have been reflected in demands for quality control personnel 
with higher analytical abilities. As a consequence, the supply of personnel 
with advanced quality control skills seidom kept pace with the demand. 
Activities in educational fields and technical societies reflected these 
changes. Technical societies created and intensified programs to promote 
professional development in quality control. Some universities were respon- 
sive to the demands for the higher quality control skills by establishing 
curricula and granting degrees (BS and MS) in this field. 


Personnel problems were also being created by the attractive offers 
tendered to Government quality control experts by industry and the rapid 
turnover in the more skilled quality control categories. This condition 
was further aggravated by the fact that because they are paid ‘under different 
pay plans, the wage board product inspectors frequently receive higher pay 
than the higher level quality control specielists.* 


In contrast with the deterioration in the Department of Defense position 
in the competition for quality contro) experts, defense contractors were 
succeeding in obtaining fully qualifiei quality control personnel to control 
and direct their programs. The disparity between the capabilities of the 
product inspector who usually represented the Government and the professional 
quality control engineer who represented the contractor in resolving mutual 
problems regarding defense contracts was more and more in evidence. Industry 
spokesmen frequently expressed concern over the need for the Government to 
be represented by individuals who could deal with their industry counterparts 
on equal terms. 


Upon initiating its review of requirements for quality control personnel, 
the Department of Defense found that three basic personnel typ .s were needed 
in its program. These were (1) production line inspectors, competent to 
carry out examinations and tests of products throughout the manufacturing 
processes and equipped with a knowledge of production methods in one or more 
commodities, (2) mathematical statisticians competent to désign and develop 
sampling procedures and other statistical quality control techniques, to 
analyze the quality control data developed, and to take appropriate action, 
and (3) quality control specialists possessing advanced training and analyti- 
cal skills, whose principal duties are concerned with directing, planning, 
and surveillance of quality control programs. In addition to a thorough 
knowledge of the techniques of product inspection, quality control specialists 
must possess competence in statistical quality control work, and in inte- 
grating activities of product inspectors and mathematicians in a total quality 
control program. 


Estimates were made of quality control personnel needs under the new 
Department of Defense quality control programs, which placed upon suppliers 


* As used in this paper the term "quality control specialist" embraces all 
of the quality control position titles established in the standard, i.e., 
Quality Control Specialists, Quality Control Assistant, Quality Control 
Representative, and Quality Control Director. 
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the responsibility for making inspections and tests and installing manufac- 
turing and quality controls, to assure the quality and reliability of their 
products. Following that, a review of the staff employed at that time 
indicated that there were sufficient numbers of product inspection personnel 
and mathematical statisticians to complement the. work of the technical quality 
control specialist types capable of adequately performing the newly reoriented 
quality control function. In view of the possible effects of the shortage on 
the successful execution of the new program, serious consideration was given 
to delaying the promulgation of unified Department of Defense policy covering 
suppliers' inspection and quality control systems. It was clearly evident 
that the new program could not be accomplished properly by relying solely 

on the personnel then in the system. This was no reflection on the capabi- 
lity of the trade and craft type inspector to do the job for which he was 
hired. After all, product inspection did not take a high degree of manager- 
ial and analytical ability. The inspector who could determine that steel 
thickness, as measured by a micrometer, was outside tolerance limits could 
not be expected, solely on the basis of the craft skill for which he was 
hired, to evaluate control procedures nor to explain to a contractor's 
management staff his reasons for deciding that the control procedures were 
unacceptable. To have demanded such abilities as a prerequisite for conven- 
tional product inspection work would have been totally unjustified. 


Despite these important considerations, it was determined that the 
unified policy should be promulgated, but that in recognition of the shortage 
of qualified personnel, sufficient time would be allowed for transition to 
the new methods to permit the recruitment and training of qualified personnel. 


Another urgent consideration which had to be weighed carefully in 
establishing the training and in setting the speed of this conversion was 
its possible effects on inspectors then on the job. Although it was 
evident and clearly desirable that the new program would result in greater 
efficiency, calling for fewer but better people, it was imperative thet the 
conversion be accomplished in such a manner that reductions in force would 
be avoided, in full compliance with Department of Defense policy. However, 
the fact that the average age of on-board inspectors was high, with many 
approaching retirement, could be counted upon to ease the situation. 


THE STANDARDS FROM THE PERSONNEL POINT OF VIEW 


The situation discussed above was further complicated by the fact that, 
historically, in Federal employment there have been a variety of pay systems. 
Two main ones have been the system applicable to Classification Act positions 
which, broadly speaking, are white collar jobs and the ungraded or wage 
board pay system applicable to blue collar jobs. Pay for the former is 
specified in the Classification Act in the form of a pay range applicable 
to each "grade." Each "grade" or level is defined in the Classification 
Act in terms of difficulty and responsibility. Salary rates applicable to 
each grade are the same regardless of geographic location. Except for modi- 
fication in order to hire for positions in shortage categories at pay rate 
steps above the minimum for the grade, salary rates for white collar jobs 
can be changed only by Act of Congress. By contrast, pay for wage board jobs 
is--generally speaking--set by the employing agency based on the prevailing 
rates being paid in private industry in the locality. Levels for wage board 
positions are not set by Congress. There are many fine points involved in 
defining the coverage of each of these major pay systems. However, precise 
distinctions need not be made for the purposes of this paper. Generally 
speaking, the Classification Act applies to white collar and professional 
positions while the wage board system is used for trade, craft and laboring 
positions. 


For many years (including those of World War II) practically all posi- 
tions in the inspection programs of the Federal Government were of the Classi- 
fication Act type and were paid Classification Act salaries. Following 
World War II, however, many Federal agencies began placing emphasis on 
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recruiting skilled craftsmen and tradesmen to staff inspection positions. 

In part this trend arose from a conclusion on the part of the agencies 

that end-product inspection required the same types of knowledges and skills 
as those required to Fabricate the product initially. In part, however, 
this trend also received impetus from the fact that higher wages could be 
paid under the wage board system and these facilitated recruiting. In addi- 
tion, the Congrees had imposed a "ceiling" on the number of Classification 
Act positions in certain agencies but had not similarly limited the number 
of wage board positions. There ensued so much confusion as to the appro- 
priate pay category of inspection positions that the Civil Service Commis- 
sion, in 1952, responded to agency requests for clarification of the pay 
system determination by issuing specific instructions and criteria for 
making this type of decision. Those instructions emphasized that the factor 
which would determine the pay system for any position would continue to be 
the nature of the qualifications required to do the work. 


The Commission's instructions could not, of course, eliminate or even 
alleviate, the salary and numerical ceiling pressures which continued to be 
pressing. It is very probable that many of the decisions on pay category of 
individual positions which were made in those years were influenced more by 
the salary and ceiling issues than by the more pertinent criteria of quali- 
fications required to do the work. Ultimately, the confusion surrounding 
the pay category of inspection positions of the traditional variety was 
joined by the problems of adapting inspection programs to the quality control 
concept. The incongruous result was that jobs were being ungraded and crafts- 
men were being hired to fill them at the very time when the needs of the 
inspection programs for a different type of personnel were increasing. 

Small wonder it is, then, that problems arose in staffing the more modern 
quality control programs which ranged on the one hand from application of 
statistical sampling techniques to end product inspection (representing a 
special phase of quality control), to, on the other hand, a very broad appli- 
cation of the full range of quality control principles and techniques during 
the entire life cycle of the commodity, equipment, or process. 


In the latter part of 1958, at a time when the agencies had pretty much 
identified the program changes to be made, Mre. Mary E. Whelan of the Civil 
Service Commission's Regional Office in Cincinnati was assigned by the 
Commission to make a study of the occupation. The purpose of the project 
was initially to deal with and produce position classification standards and 
qualification standards for only a relatively few positions in the inspection 
group. Other types of positions in the group were to be covered later. 


At the invitation of the Civil Service Commission, the Department of 
Defense and other agencies participated actively in the development of the 
standards. This not only permitted but also encouraged and speeded a review 
of the impact upon personnel requirements of the fundamental changes in 
the quality control program. This study required intensive joint effort by 
agency quality control and personnel managers. 


The Department of Defense established task groups to study various 
aspects of the problem. Personnel and quality control managers in the Army, 
Navy and Air Force, and within the Office of the Secretary of Defense parti- 
cipated. Counterpart staff representatives in the General Services Adminis- 
tration and other interested Federal agencies were informed of the work of 
these groups, and cooperated in the formulation of the standards. 


Four months of factfinding included travel to many Government installa- 
tions and industry facilities. Among the activities visited were those of 
Army, Navy and Air Force in the Department of Defense, General Services 
Administration and other non-military agencies. A greater amount of time was 
spent in the military services since the largest proportion of Federal 
employees engaged in this work are Department of Defense personnel. 
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Information gathered during the factfinding revealed that partial 
treatment was impractical. Therefore it was determined to enlarge the 
scope of the project to formulate a single master standard applicable to the 
entire inspection occupational group (with certain limited exceptions). 
This was based on the conclusion that a common core of knowledge require- 
ments, duties and responsibilities that permeated the entire group had been 
successfully identified. The distinctions that marked the specializations 
in the gropp were commodity knowledges that served to identify differences 
in kind of work and knowledges required. However, elements which affect 
the levels of difficulty and responsibility were similar from one commodity 
specialization to another. 


One of the major conclusions of this analysis was that quality control 
represented a major career field. Product inspection was recognized to be 
an important element of quality control but to constitute only one phase 
of the overall quality control program. 


The standards in existence at the time the study was launched consisted 
of a loosely related collection of standards applicable only to conventional 
inspection type jobs. Numerous problems were involved in attempting to 
develop a completely integrated coverage for both those inspectors and the 
more demanding quality control specialists. Those problems extended to 
(1) what commodity specializations needed to be identified, (2) how the 
internal series structure should be organized, (3) whether the several 
functional specializations (e.g., those for inspectors and quality control 
specialists) should be in the same series or in parallel series, and (4) 
how the title structure should then be established. 


On the basis of this study, the Standards Division of the United States 
Oivil Service Commission developed a complete proposal for review and comment 
by the Commission's regional offices and the Federal agencies. Upon 
receiving these comments, the Standards Division undertook to evaluate 
the comments, to reconcile differences in points of view. In many instances 
the differing reactions stemmed from variations in the degree to which the 
quality control program had been implemented in various organizations. In 
the main, the reaction was favorable although many changes were suggested. 
Nevertheless, some agencies expressed dissatisfaction with the proposal. 
Careful consideration was given to the opposing points of view. Upon 
assessment of all the comments and incorporation of constructive suggestions 
and changes based on valid criticisms, it was decided that issuance of the 
standard as an official document would be a sufficient improvement over the 
existing standards to proceed without delay. 


One of the important objectives in the preparation of the standard was 
to improve the criteria for distinguishing between Wage Board and Classifi- 
cation Act inspection positions. A thoroygh and comprehensive set of dis- 
tinguishing criteria was established in the new standard. No flat and 
arbitrary rules were laid down. However, it has been generally recognized 
by quality control and personnel officials that these have been made as 
specific as they can be. Basically the guidelines provide for and require 
thorough analysis and judgment prior to arriving at a decision regarding 
coverage. The nature of the work, the qualification requirements, the type 
of program to be carried out, which in turn determines the skills and know- 
ledges required, significantly influence the ultimate decision. 


In addition to the classification standard which covers the evaluation 
of the kind of work and the level of responsibility and difficulty, the 
Civil Service Commission developed a new qualifications standard which sets 
forth the skills, knowledges, abilities, and other qualities required to be 
eligible for these positions. As i8 now customary, both the classification 
and qualification standards for quality control and inspection positions 
were developed at the same time and by the same standards writer. The under- 
lying premise of the qualification standards was based on the demonstrated 
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evidence that quality control personnel must have skills, abilities, and 
knowledges over and beyond those required of inspectors and inspection 
specialists. It was recognized that there would be a major problem in 
selecting, training, and converting those inspectors who could meet the 
higher requirements of the new quality control assignments when the shift 
in the program and operations was accomplished. In view of the differences 
in the job demands, it was necessary, therefore, to establish one qualifi- 
cation standard for the quality control type position and a different one 
to cover the inspection type. It was also recognized that although the 
ungraded type of inspection work is acceptable experience for the graded 
inspection positions it is not qualifying for quality control positions. 


A major source of recruitment for quality control positions was 
expected to be from among college graduates or applicants with experience 
who qualify under the Federal Service Entrance Examination. The Federal 
Service Entrance Examination is a very broad examining vehicle used to fill 
entrance-level positions in a great variety of management, administrative, 
investigative and professional occupations. In lieu of réquiring Federal 
employees already in Classification Act inspection positions to pass the 
written test for the Federal Service Entrance Examination, the standard for 
quality control positions provides for and encourages the development of 
agency programs designed to screen and train inspectors in order that they 
can qualify for the new quality control positions. 


SS OF P S_AS QUALITY CO L GEMENT SEES IT 


In the opinion of quality control program managers, the Civil Service 
Commission has been successful in constructing a standard which is fully 
abreast of the times and adequately reflects the present state of develop- 
ment in the field of quality control. The standards reflect advancements 
in product complexity and quality assurance technology which have occurréd 
in recent years. They provide a suitable progression in levels (and permit 
the establishment of increments in salary compensation that give proper 
recognition to the higher skills). The standards facilitate recruiting 
urgently needed personnel with higher potential and skills. At the same 
time, they permit the conversion of experienced on-board employees who 
have suitable qualifications to the new quality control positions, and reten- 
tion and utilization without undue dislocation of those who remain inspectors. 
The continued existence of some inspection positions under Classification 
Act coverage provides an additional benefit by serving as a buffer to protect 
seasoned personnel who may not qualify for the new occupational specialty, 
or who may not convert for other reasons. 


But to the quality control program manager, the principal achievement 
of the new standards unquestionably is its recognition of quality control 
as a new occupational specialty requiring a high level of abilities and 
skills. Without this far-reaching realignment the task of conversion to the 
new quality control concepts would have been far more difficult or else 
completely impossible. Quality Control program officials believe that the 
new occupational standards contribute significantly to the accomplishment 
of the quality control program of the Department of Defense. 


SUMMARY 


A number of conclusions can be drawn from the facts that have been 
presented. Both in Government and industry, quality control has attained 
widespread recognition as a highly specialized technical field meriting the 
establishment of one or more new occupational fields. The quality control 
policies and programs of the Department of Defense, the General Services 
Administration and other Federal agencies have had a significant influence 
in the rapid advancement of quality control engineering and technology, 
both in the United States and abroad. The new classification and qualifi- 
cation standards for quality control and inspection position constitute 
another significant contribution by the Federal Government in encouraging 
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advancements in this specialized field. 


There is a widely held opinion in the agencies that for the immediate 
future requirements for quality control engineers will constitute only a 
minor proportion of the personnel engaged in quality control activities. 
In the Department of Defense, quality control specialists and representa- 
tives will probably constitute the major portion of its inspection and quality 
control staff. The number of mathematical statisticians in the quality 
control organizations should remain essentially unchanged. There will 
probably be a gradual decline in the number of wage board type inspectors, 
especially in procurement programs. The number of Classification Act type 
inspector and inspection specialists can be expected to decline to some 
degree in future years. 


There is some difference of opinion regarding the number of professional 
engineers required in the quality control program, what their role should 
be, and what needs to be done to identify better to career patterns. In the 
Department of Defense professional engineers have been utilized in increasing 
numbers. Some quality control program officials favor a further increase in 
the number of professional engineers utilized in the program. Other officials 
favor further study of the role of the professional engineer in the program 
and also a continuing examination of the career opportunities available to 
professional engineers who are already in the field of quality control. With 
regard to the occupational identification of the professional engineer work- 
ing in the field of quality control, many program officials agree with the 
Civil Service Commission's recent decision not to identify quality control 
engineering as a separate engineering discipline. Inherent in that decision 
is the concept that quality. control is a specialized function which may 
exist within any of several of the established engineering disciplines. For 
example, in the position classification standard for the Industrial Engineer- 
ing Series, GS-896-0, quality control is described as one subject which may 
involve knowledges and abilities characteristic of industrial engineering. 
There are other quality control program officials who would have preferred 
to have quality control engineering separately identified. Since quality 
control is still evolving, it is yet too early to predict precisely how 
professional engineers working in the field of quality control will ulti- 
mately be identified and utilized. Continued study of these aspects will be 
required of both program and personnel officials in order to keep pace with 
the total development in quality control. 


THE JOB AHEAD 


The personnel standards which recently have been developed reflect the 
current status of development of the occupation. Quality control is a dynamic, 
constantly changing program. Therefore, the demands placed upon those who 
carry out the quality control functions are still changing and will continue 
to change. It is not clear at this time what the nature of the changes will 
be. The immediate job ahead is the achievement of a smooth transition from 
the previous product-inspection-oriented to the present quality-control- 
oriented programs. Care and prudence must be exercised during this transi- 
tion period to insure that human considerations remain paramount and that 
full assurance of quality of defense and other Federal materials is not 
compromised. The long-range job ahead is the continuing responsiveness to 
changes in the program which in turn will determine the ways in which quality 
control positions evolve. 
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SLICING THE PIE 


Cdr. J. A. Barker, Director, Assurance Systems Engineering Division 
Department of the Navy, Washington, D.C. 


For a variety of reasons there has been recently a tremendous upsurge 
in the emphasis on reliability. Some of the reasons are economic, some 
relate to mission success, some relate to maintenance and logistic aspects. 


Out of this upsurge of emphasis have sprung many facets of effort in 
reliability. The major ones appear to be: 


(1) The treatment of reliability as a measurable quantity and expres- 
sions of measurement in numerical terms. 


(2) Inclusion of quantitative reliability requirements in purchase 
specifications and contracts. 


(3) Apportionment of a reliability requirement to the subportions, sub- 
systems, units and parts of a product as definitive design parameters for 
these elements. 


(4) Techniques, procedures and tests for demonstrating, on paper and 
physically, the degree of reliability and the confidence level that have 
been achieved in the product during the various stages of its genesis. 


The Bureau of Ships subscribes to the utilization of a quantitative 
treatment of reliability. Anytime that the technical aspects of a task can 
be treated in quantitative form, the result is to minimize the variability 
of interpretation amongst people working on the task. 


If we look at the problem of reliability, we find that one cannot merely 
specify the reliability to be achieved; one must also specify the extent to e 
which the achievement is to be proven; i.e., the confidence level, and the 
confidence interval over which that level holds. Implicit in this definition 
of confidence level and an associated interval is the assumption that there 
is a sufficient combination of samples to be tested, and time for test, so 
that demonstrated achievement of the specified reliability will be factual. 


This approach is sound for the relatively high volume products with 
mean times between failures in the hundreds of hours. It is reasonable in 
terms of economy, inventory requirements, delivery schedules and other con- 
siderations to expect that a sound statistical verification of product relia- 
bility will be made prior to placing the product in service. This is sig- 
nificant because this is what the contractor signed up to do. True, there 
may be further requirements that the contractor follow the product in service 
as a design improvement process, but speaking generally on this class of 
product, a contractor's responsibility is to deliver a product shown to te 
satisfactory by test. 


At the other extreme of the technical gamut, lie those complex equip- re 
ments of which only a few will be produced. In addition, the desired fail- iE: 
ure-free time in operation often runs up into the thousands of hours. Veri- F 
fication of product reliability by appropriate combination of test samples 
and test time becomes prohibitive in terms of cost and time before the pro- 
duct can be placed in service. 


So this then is the dilemma for the low volume product. It is obvious 
that we would like to have a product with high reliability proven to a high 
confidence level over a small confidence interval, but verifying fully by o.. 
test that these objectives have been reached is impractical. Tests, inspec- 
tions, reviews of subportions or even prototypes, only take us part way 
towards the specified confidence level. 
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How do we bridge the gap between a factual confidence level and a 
specification requirement? Whether we like it or not, or whether we find 
it sufficiently quantitative, we do attempt to bridge the gap by indirect 
means. 


A contractor asserts that he has bridged the gap by presenting those 
aspects of his operation that should give the customer the necessary con- 
fidence. The contractor describes the significance of partial tests and 
the reliability predictions; shows the consideration he has given to prob- 
lems; points out the capabilities and past successes of his personnel. 


The customer evaluates the contractor's presentation. If he is con- 
vinced that the desired level of confidence has been reached, he accepts 
the product. If not convinced, the customer may require further partial 
testing; use of consultants, additional analysis. 


Erroneous estimates of the indirect. confidence, by either party, have 
serious implications. Yet the estimates are very difficult to make accur- 
ately because so many intangibles are involved. 


The importance of improving the ability to estimate indirect confidence 
warrants full use of all approaches that will remove some variability from 
the estimates. 


A major effort in this direction is in the development of systems to 
provide planned and comprehensive assurance of conformance to contractual 
requirements, up to the limits of physical testing mentioned previously. 
Quality Control Systems, as defined in MIL-Q-9858, encompass conformance 
in manufacturing. Reliability Programs, Reliability Assurance Systems, 
Reliability Control Systems, generally encompass the engineering decisions 
and conformance. 


The display of the scope and depth of these systems, and the informa- 
tion derived from their use, is the contractor's mechanism for showing he 
has bridged the confidence gap. The evaluation of these systems is the 
customer's mechanism for assuring himself that the sum of the confidence 
level derived from test and the amount of indirect confidence add up to 
the required confidence level. 


The use of the indirect confidence places special emphasis on the 
reliability of the people who in many ways contribute to the reliability 
of the product. What is the probability that the people; engineers, crafts- 
men, buyers, stock clerks, supervisors, will perform satisfactorily for a 
specified period of time in their physical and mental environment? This 
is a basic question for any production job shop. It is also a basic ques- 
tion for most engineering design groups, for design is frequently a job 
shop type of operation; each job generally having significant differences 
from previous work, and the number of designs done repetitively is small. 


Attempts to improve the reliability of people extend over a long period. 
No real single factor has been found which automatically reduces the varia- 
bility and unpredictibility of the actions of people. Variations are caused 
in engineers, in craftsmen, supervisors and fork-lift operators by many 
factors. These range all the way from domestic problems, to encounters with 
new frontiers where the approach to a problem is at. best only an educated 
guess. It should be kept in mind that any person contributing to the product 
can, at any time, make a mistake which partially or completely destroys the 
effort and expense that has gone into building a reliable product. 


The successive re-cycle of design, production, test - design, production, 
test - design, production, test has been the traditional method of arriving 
at a reliable product. However, in the areas where this is impractical, the 
focus must be on the reliability of the people. 
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There are no new magic tools which will suddenly make people reliable. 
Quantitative treatment and prediction of reliability are a big step in this 
direction. There are, however, a few tested standbys which will help to 
remove some of the factors that lead to unreliability and thus in the nega- 
tive sense, contribute to improve reliability. These factors are the follow- 
ing: 


(1) Organizetion. In the literature there are articles describing the 
functions which have been assigned to special groups such as reliability 
engineering, quality control or various other names. Over and over again it is 
obvious that if all the assignments are carried out, then the special groups 
are supermen; authorized to review and change designs; to contribute as part 
of an engineering team to reliability; to be father confessors to any relia- 
bility problem. An assignment of duty such as this makes these groups both 
judge and jury. It makes them authorities without responsibility. It 
beclouds what should be proper and clear lines of: 


(a) authority for decisions 

(b) responsibility for decisions 

(c) responsibility for conformance to decisions 

(da) responsibility for assuring that the decisions and the conform- 
ance lead to a satisfactory end product. 


The lack of clear cut lines of authority and responsibility can only 
lead to gaps in the coverage of the problems that must be solved; to shrug- 
ging off of responsibilities because no one really knows who has it; to 
internal conflicts of professional and trade cognizance. 


(2) Physical and Procedural Environment in Which Personnel Operate. The 
physical environment reflects the compatibility between the sensitivity of 
the work being done to foreign material or contamination, and prevalence of 
such contaminants in the areas. It is unreasonable to expect a craftsman 
to keep his work free of contamination when he is surrounded by dirt; when 
his tool for cleanup is an air hose with which to blow out chips; when the 
transportation of subportions of the equipment through the plant leads to 
exposure to dust, trash and rain. That is obvious in the production area. 
Less obvious in the engineering and production areas is the procedural 
environment. Are there current copies of Military Specifications readily 
available in design? Are there definitive instructions concerning the size 
of lettering to be employed in drafting, so that reduced size plans will 
still be legible? Is there a formal effort by management to make sure that 
the engineer, the craftsman, even the janitors, are aware that management 
is interested in a reliable product? Are the problems associated with 
reliability outlined as tasks, and assigned and ticked off so that there 
is a knowledge of what has been done and what remains to be done? Is there 
an internal agreement that vague specifications by the customer, such as 
"free from injurious defects" will be interpreted in quantitative terminol- 
ogy such as "defects less than 5% wall thickness, fair out; less than 12% 
wall thickness, fair out and weld; greater than 12% wall thickness, discard". 


(3) Analysis o uct Interfaces. A lot of factors cross the inter- 
face between the contractor's product and the surroundings. Electrical 
power, cooling water, vibration, shock, dust and dirt, people, control sig- 
nals; all enter the product from outside sources. Analysis of these inputs 
for completeness in regard to desirable inputs, and for undesirable inputs-- 
indentification as problems; should make it much more likely that the designer 
will consider what he must do to correct for inadequate or unwanted inputs. 
Here again if the customer has not defined the nature and range of these 
inputs, there should be an internal mechanism by the contractor which will 
set these ranges, clearing these as necessary with the customer. 


(4) Rec tion of lications. As the engineering and production 
of a product gets more and more subdivided into numerous specialties, it 
becomes harder and harder for any one individual to recognize the significance 
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of his decisions, or work, in terms of the implications of these decisions. 
There is need for a formal campaign which examines the requirements coming 
in for a new product and singles out the new aspects. These aspects may be 
to use stronger steel, less humid air, nonflammable lubricants, more accur- 
ate measurements. Normally these requirements would be translated by engin- 
eering into plans and specifications. Then they would be transmitted to 
Production. At this time it may be learned that Production is not prepared 
with facilities, with training, with understanding, to meet the new require- 
ments. There is need to point out early to all personnel just what the 
implications are of the new developments. In particular, there is need to 
provide throughout the organization an agreement on which features of the 
new processes or requirements must be factually proven correct; which must 
be proven correct to a lesser degree, and which are of such a nature that 
reasonable assumptions about their correctness are sufficient. In this same 
connection, magazine articles and presentations often convey the theme that 
the only thing that can happen to a design once it leaves the engineering 
group is to have it degraded by everyone else. While it is very true this 
is often what haprens, it is also true that a good many poor designs are 
improved upon by production recommendations. If there is to be an overall 
improvement in the reliability of people contributing to indirect confidences 
in the product, then there must be recognition that the engineer, the crafts- 
man, the stock clerk, the truck driver, the contract man are each links in 
this total network. Assertions of superiority by any one group can only 
lead to the attitude by others, "if they're so smart let them figure out 
what to do about this mistake they made." 


The procurement dollar is often presented as a pie, with slices of this 
pie apportioned to the many places. In cutting a slice for reliability, par- 
ticularly for the low volume,long MIBF products, there must be a share saved 
for those areas that are important to improvement in the relisbility of the 
people creating the product. 


The opinions or assertions contained therein are the private ones of 
the writer and are not to be construed as official or reflecting the views 
of the Navy Department or the naval service at large. 
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WHAT'S THE DIFFERENCE? 


Mason F, Wescott, Professor of Applied Statistics, 
The Statistics Center, Rutgers-The State University, 
New Brunswick, N, J, 


INTRODUCTION 

The technique presented in this paper is simply a graphical interpretation, 
based on testing differences between means, for a standard 23 factorial with 
replication design, It has the unique advantage that its conclusions can be repre- 
sented visually in a form close enough in appearance to a control chart to be easily 
and meaningfully understood by anyone familiar with the control chart concept, This 
is an important advantage because it serves to simplify the sometimes difficult task 
of communicating to production and plant personnel not familiar with the meaning of 
formal experimental designs the practical interpretations buried in these designs, 
In addition, this graphical technique has the attractive property of being easy to 
formate and fast to carry through in a wide veriety of actual production situations, 


STRATEGY AND DESIGN 


The ingredients needed to make the technique applicable are: 

(1) A problem which the people concerned can agree involves one dependent (or 
"response") variable and three independent variables (or "factors") each of 
which can be replicated at two levels, The independent variables would be 
called "main effects" in experimental design, They (and the levels chosen for 
them) must be agreed on prior to setting up the shop (or laboratory) situation 
for getting the data, and they must be variables (and levels) of practical 
importance, 

(2) Facilities mst be available (and strict supervision exercised) to make sure the 
data are really obtained as reouired by the design, 


The design (or "blue-print") for getting the data may be laid out as shown in 
Figure 1, This is clearly a straightforward 23 factorial design, Numbers in paren- 
theses merely serve to give cell identification for describing the analysis procedures 
to be followed, 


Factor 1 Level 1 level 2 
Factor Level Level 2| Level 1 |Level 2 
ay (5) (7) 


FIGURE 1 - Design Layout for Getting the Data 


There will be n observed values of the dependent (response) variable in each of 
the 8 cells, a total of 8n observations altogether, For each of the 8 cells, we 
shall need to compute two statistics: the cell mean and the cell range, if n £10 
or the cell standard deviation ifn >10, Thus %4, Ry (or @4) represent the required 
statistics for the i-th cell, 


We shall want to test the 8 ranges (or standard deviations) for statistical 
control, and if control exists, to estimate o, from R (or c) in the usual way, i.e, 


R/dy (or G/e2). 


There will be 7 "comparisons" we shall want to make and exhibit graphically, each 
comparison being simply the difference between the of ‘some of the 6 3 
the mean of the remaining cells, Thus, D,; represents the kl-th one 


the 7 comparisons required, Three of these comparisons will be for main effects, 
three will be for first order interaction effects, and one will be for the second 
order interaction effect, 


All the observed values of the dependent (response) variable when test conditions 
are set up for (Factor 1, Level 1) and (Factor 2, Level 1) and (Factor 3, Level 1) 
are recorded in the cell labeled (1). (See Figure 1). When test conditions sre set 
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up simulteneously for (Factor 1, Level 2) and (Factor 2, Level 1) and (Factor 3, 
Level 2), the observed values of the dependent variable are recorded in the cell 
labeled (6), Similar considerations dictate the sources from which data for the 
other cells will come, 


The number n of observations ("replications") to be recorded in each cell will 
be determined by the number estimated as necessary to get the desired discrimination 
between any two of the seven pairs of means X%,,%), or it may simply be some arbitrary, 
convenient number which will facilitate getting the data ovickly and with the minimum 
of interruption to routine operating conditions, In the former case, an estimate of 
the standard deviation of the dependent variable must be available from sources out- 
side the actual data of the "experiment" (the "sigma-known" case). Also, in this 
case, the critical difference between any two comparisons (X,vs X,) which it is 
desired to have the experiment pick up at a preassigned level of significance mst be 
specified, In the latter case (n arbitrary and "sigma unknown", i.e, computed from 
the data of the experiment), the magnitude of n is deterrined purely by "practical" 
rather than "statistical" criteria, 


Suppose @, is an independent estimate of Cy, and suppose Dig *k-k1, is the criti- 
cal difference we want to pick up with significancea, Then n may be found by | 
solving for n in the relation (Dyy) 2n/6, = Zea , where is the normal deviate 
corresponding to a two-tail significance level of %, 


After the data have been collected, calevlate X and R (or o,) for each of the 8 
cells, and make a range (or standard deviation) control chart to see if there is 
enough evidence of homogeneity of variances to continue the analysis, If there is 
serious out-of-control evidence on this chart, any further analysis is open to 
question and the experiment should be repeated or assignable causes sought to explain 
the lack of homogeneity in cell-to-cell variability--or both! If the variances can be 
regarded as sufficiently homogeneous, estimate oy from (or where and 
Cp are the usual correction-for-bias constants used in routine control chart studies, 


Next, calculate = o'/W2n, where has been estimated from (or o/c) 

age R s the r of observations in each cell, Now find X for all 8 cells from 
4 or 2X4/6n) and set up what looks like a control chart with central line at 
%, “decision lines" at t+ ho), and a "sample number" scale that will provide for 
plotting 1h points, The value to use for the mltiplier h in setting up the decision 
lines will depend on whether one uses o estimated from — (or co), or the i 
estimate @,, in computing o,. When ot is used, h = sta (8n-8) where tq (8n-8) is 
the Student t value at (8n-8) degrees of edom corresponding to a two-tail signifi- 
cance level ofa. When oy is used, h = where Za is the normal deviate corre- 
sponding to a two-tail significance level of. Generally spesking it will be more 
; conservative to use the t form for h, altho a "quick-and-dirty" velue that may suffice 

. is simply h=1, If there are no border-line points, i.e, no plottings close to the 
decision lines thet go with h*1, these lines will suffice to provide valid decisions, 


Altho the procedure for completing the analysis could be described in general 
terms, a specific numerical example will be easier to follow, Data are from an 
actual case history application but have been coded, and factors and levels have been 
left undefined to protect source identity, Note that the two levels of Factor 3 are 

Pe attribute rather than variables levels, 


A CASF HISTORY EXAMPLE 

Serious difficulties were being experienced in maintaining uniformity in a 
response variable "length" for product coming from a particular type of machine, It 
was agreed that among a list of factors believed to have an effect on this Agnath 
variable, three stood to be sufficiently important to be worth trying in a 2 
factorial experiment, These factors may be identified as M, W, and T and the coded 
values of the levels chosen are shown in Table I, Shown also in this table are the 
mean and range (in coded values) for each of the 8 cells; the individual observations 
within each cell are not show, 
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TABIF I - Data for Case History Example 
w "86 766 Wy"92 


From some scattered control chart data prior to this experiment, it was estimated 
that if control could be maintained a ot of about 7 could be expected, Hence, 


6,°7 was a "known-sigma" value, It was desired to have the experiment pick up as 
significant at the 5% level a difference of l between any two means Foy So 
Z_ 2, and we have |, which gives Since deta were 
fast and easy to get from this process, n=10 observations per cell was chosen, 


From the data of Table I it will be found that R=26.50, 1Cl_*h7.08, ICl_"5.92 
and variation is random within the control limits, Hence, o1=26,50/3,078=8,609, so 
= 1.925, Also = -15.7/8 = -1.9625 and o¢(72)*1.99h, Hence 
h=1,99i/2 = .997, and decision lines are at -1,9625 + (.997)(1.925)=-1,9625 + 


1.9192*-,0h3 and -3,882, We may now lay out the chart on which we shall plot the 7 
comparisons as soon as we have them computed, 


To compute these comparison values, we build up tables like Tables II and IIT 
from the basic data of Table I, Plot on the chart (see Figure 2) the three main 
effect comparisons from the entries in the third column of Table II and the four 
interaction effect comparisons from the entries in the third colum of Table III, In 
doing this, label the "sample number" points (from left to right): 

(1-2-3-h) 5-6-7-~8) 3( 1-2-5-6) =( 3( 1-3-5-7)-( 2--6-8) (1-2-7-8) 5-6) 5 
(1-3-6-8) 3 Join by a straight line 
each of the plotted points corresponding to the respective hyphenated pairs given here, 


Any of these 7 lines that more than spans the decision lines on the chart repre- 
sents a comparison between the two levels of the associated effect represented by the 
line that is significant at the 5% level, i.e, there is a 1 in 20 chance that we will 
be claiming as significant a difference that is actually just an urmmusual sampling 
variation, Conversely, any of these 7 lines that fails to span (is included entirely 
between) the two decision lines represents a comparison that can be regarded as a 
sampling variation, We shall not try to pin down the magnitude of the Type 2 error 
committed by this latter decision, but it is of the same nature as the decision we 
regularly make to "leave the process alone" when points fall within control limits on 
an ordinary control chart, 


The completed comperison chart (Figure 2) shows that none of the interaction 
effects are significant at the 5% level, tut that the two levels of factor M are 
highly significant and that the two levels of factor W ere also significant, 


COMPARISON WITH ANOVA 
If we take the data in Table I and run a standard analysis of variance for a 
23 factorial with replication, we will obtain the results shown in the Anova Summary 
Table exhibited as Table IV, Note that the conclusions are identical, viz no signif- 
icant interaction effects, but factors M and W are significant as main effects, with 
M showing significance at the 1% and W at the 5% levels respectively, 
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TABLE II - Averages for Main Effects Comparisons 


Factor Level Cell Numbers Average Required 
= 0,650 
My 5-6-7-8 = 
Wy 1-2-5-6 = -3,975 
Ww 
Wo 3-7-8 Two = 0,050 
H 1-3-5-7 = -1.450 
T 
s 2-h-6-8 = -2,475 
TABLE III - Averages for Interaction Effects Comparisons 
Interaction Cell Numbers Averages Required 
1-2-7-8 (MW), = -3.150 
3h 5-6 (Fa), -0.775 
1-3-6-8 (MT), = -3.325 
(RT), = -0.600 
1-4-5-8 (WT), = -2.050 
2-3-6-7 (WF), = -1.875 
1b-6-7 -3.175 
2-3-5-8 (WT) >" -0, 750 


& 
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FIGURE 2 = Comparison Chart for Data of Tables I-II-III 
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TABLE IV - Anova Summary Table for Data of Table I 


Source Sum of Squares a.f, Mean Square Fe 
5h6,01 1 546,01 
32h,01 1 32h,01 
21,02 1 21,01 
122,81 1 122,81 1,58 
MxT 18.51 1 148.51 2,08 
wer 0.61 1 0,61 
MXWxT 117,61 2 117,61 
Error 32 72 721.45 
Total 641,89 79 


F og(1,72) 3.97 o(1,72) = 7,00 


Note also that the error variance in Table III gives #= { 71.45 = BLS vs 
R/dp = 8.61, (If standard deviations per cell in Table I are used, it turns out that 


G/ep = 8.49 which is even closer to the anova estimate of 8,5.) 


Tms the graphical analysis and the standard anova strategies lead to the same 
conclusions, tut the computational details are somewhat heavier in the latter pro- 
cedure and the naked F-test statements lack the sex appeal that is visible thru the 
graphical presentation of the comparisons involved, As a fast-moving tool that gets 
answers which make sense to plant personnel, this little technique is well worth a 
trial if it is important to know "what's the difference" without going into a full 
scale analysis of variance, 


REFERENCES 
1, “A Production Experiment "sing Attribute Data"; Ennerson, Fleischmann, and 
Rosenberg; IQC, March 1952, p, hl, (Unfortunately there are several errors and 
misprints in this paper that the reader will have to discover for himself, ) 


Fe pga Experiment with Mechanical Assemblies"; Fllis R, Ott; TOC, May 
1953, Pp. 


3. "Use of Edge-Notched Cards in the Analysis of Variance"; Lloyd S, Nelson; IOC, 
April 1957, Pp. S. 
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PRACTICAL APPLICATIONS OF NORMAL PROBABILITY PAPER 


Robert Cardno 
Supervisor - Quality Control 
Ward Street Products 
Canadian General Electric Co. Ltd. 


After listening to a particularly fine talk on a Quality Control 
technique, how often have we heard members of our section make these ob- 
servations: 


"Yes, it was a fine talk, and the speaker obviously knew his subject from 
A to Z. The theory he set forth is doubtless OK, but he didn't show us 
where it has been used as a practical day to day tool”. 


In other words, we are willing to accept theories provided by our long 
haired friends up in cloud 17, but how about giving us something practical 
that is being used, and which we can apply to our everyday operations. 


It is with this idea in mind that this talk is being given to-day. 

You can be assured that anything you hear to-day will be practical because 
it is being given by a practical person who is engaged in applying statis- 
tical methods to the control of product quality in a manufacturing plant. 

To-day some practical uses of normal probability paper will be presen- 
ted. The development of probability paper, and the theory behind it, will 
be avoided to allow more time for practical applications. 

This is not the only method of applying probability paper to the con- 
trol of product quality. Other shorter, and perhaps better methods have 
been developed, but it is hoped by the end of this talk that you will see 
that this method does work for us. We also hope you will see where this 
technique can be applied towards the solution of problems in your own oper- 
ations. 

Now these questions may arise: 

1. What is normal probability paper? 

2. What is probability? 

3. Where can I use probability paper? 

4. What can the use of probability paper do for me? 


5. How reliable are the results derived from the use of probability 
paper? 


6. Is probability paper costly to use? 


1. What is Normal Probability Paper? 


Normal Probability Paper is graph paper with a normally spaced vertical 
scale for the measurements and a horizontal scale of cumulative percent- 
ages, so spaced that values plotted from a normal distribution will fall 
along or about a straight line. 
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What is Probability? 


Probability is simply the likelihood or chance of occurrence. We can 
say with a great deal of assurance that in all probability our Society 


will continue to grow both in number and in its ability to serve indus- 


try. This we can understand, because after all we have completed some 


16 years of steady growth. However, can we say with the same assurance 


that in all probability a manufacturing process will or will not pro- 
duce parts within specified tolerance by examining a small sample of 
the product at the start of a run. This is indeed a tall order, but 
Normal Probability Paper is a Quality Control tool which will assist 
you to do just that, as we will try to show in the practical demon- 
stration. 


Where can I use Normal Probability Paper? 


In general, probability paper can be used wherever variables data can 
be collected and recorded. The practical examples will, we are sure, 
answer this question to your satisfaction. 


What can the use of Normal Probability Paper do for me? 


A. Show if the parts measured come from a normal distribution. 

B. Determine X or average value. 

C. Determine standard deviation (c-) without extensive calculation. 
D. Make comparison with specification values. 


E. Predict percentages of production below or above any particular 
value. 


F. Assist in machine and process capability studies. 
G. Present information graphically. 

H. Provide a tool for trouble shooting. 

I. Indicate homogeneity of the product. 


How Reliable is the Probability Paper Technique? 


We will let the demonstration provide the answer to this question. 


Is Probability Paper Costly to use? 


Provided it is necessary to have the information obtained by the use 
of Probability Paper, I know of no less expensive method. This will 
also be shown in the demonstration. 


let us consider where we can use Normal Probability Paper. 
A. To check the quality of incoming material and components. 


B. To check in process quality. 


C. To check machine or process capability. 
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Certain conditions must be met if we are to make successful applications 
of Normal Probability Paper. 


1. In the case of lot or batch inspection: 


A. Samples must be taken at random. 


B. The lot or batch must be homogeneous. 


(Probability paper will very quickly indicate a heterogeneous 
product.) 


2. In _the case of “in process” quality checks or machine capability checks: 


A. Measurements must be made on consecutive parts from the process. 


B. The set-up must not be changed in any way while the sample is 
being taken. 


(Should it become absolutely ry to change the set-up, we 
must start sampling again when the new set-up has been satisfac- 
torily completed.) 


3. In all cases: 


A. The measurements must be made accurately. 


B. The measurements must be taken by one and the same person using 
the same measuring equipment throughout the entire sample to 
avoid introducing a known assignable cause of variation into the 
study. 


Now let us get on with the practical applications. -- In the first place 
we will go through the mechanics of recording, calculating, plotting and 
analyzing data. In this way it is felt that you will get a much better 
grasp of the technique than you would if it was just explained to you. 


"DEMONSTRATION" 


That concludes the demonstration of the probability paper technique as we 
apply it at Ward Street Works, Canadian General Electric Co. Ltd. 


We trust we have demonstrated successfully that this technique will do 
all that we claimed it would do at the beginning of this talk. The more we 
use this technique, the more it amazes us by its accuracy, reliability and 
economy. Doubtless, you have some questions you would like to ask, and I 
will do my best to give you the answers or get the answers for you if I 
cannot give them personally. 
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CANADIAN GENERAL ELECTRIC CO. LTD. 


WARD STREET PLANT 


Part No. or 
Material Size 


Name of Part or 
Type of Material 


Dimension Checked 


Vendor 
or Department 


44828 Pt.2 


Binder Head Screw 


Head Diameter 
-307" to .327" 


All Nu Screw Co. 


SAMPLE NO. 


SAMPLE SIZE 25 


CHECKED BY: 


DATE: Jan. 7/58 


ef % cf 


MidPoint 
Remarks 


; 
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QUALITY CONTROL CANADIAN GENERAL ELECTRIC CO. LTD. WARD STREET PLANT 
44828 Pt.2 Binder Head Screw t—. sage All Nu Screw Co. 
SAMPLE NO. SAMPLE SIZE 25 CHECKED BY: "Cc" DATE: Jan. 7/58 
x x-¥ | «x-%? CALCULATION FOR X AND o~ 

1 .314 -1 1 

2 .316 +1 1 x 

3 .316 1 = 

4 -313 -2 4 = .315 

5 .320 +5 25 

6 -315 

7 -315 

8 -316 +1 1 

x- 

10 -314 -1 1 = 

ll -315 = J3.7 

12 -314 -1 1 = 1.9 = .0019" 

13 .319 +4 16 

14 -313 -2 = .315" + .0057" = 

15 .316 +1 1 -3o0% = .315" - .0057" = .3093 

16 -1 1 

17 -315 0 

18 .314 -1 1 

19 -315 0 

20 -315 0 

21 -311 - 4 16 

22 -312 - 3 4 

23 -316 +1 1 

24 .316 +1 1 

25 317 +2 4 

X 7.886 E(x-X) 259 
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CHOOSING A SEQUENTIAL TEST 


N. L. Johnson 
Case Institute of Technology 


1. The basic philosophy of 'sequential analysis’ is now becoming fairly well estab- 
lished among both theoretical and experimental statisticians. The latter are 
becoming accustomed to include sequential methods among the possible ways of attack- 
ing their problems; the former are producing an increasingly varied set of available 
techniques. 


By ‘sequential methods' we mean methods of statistical enquiry which do not have 
a rigidly pre-determined pattern (as, for example, the classical ‘experimental de- 
signs') but which contain provisions for re-orienting, or concluding, the investiga- 
tion according to rules operated by the data as they are collected. It can readily 
be appreciated that by this means it should be possible to avoid the waste involved 
in completing a pre-determined program when it is reasonably clear, at a relatively 
early stege, what the final conclusions will be. 


In the technical application of sequential methods there are as yet a number of 
points which are of some importance but do not always receive the attention they 
deserve. They arise, mainly, from the fact that the expected behavior of a sequen- 
tial procedure depends on the real physical situation, not only in regard to decisions 
reached, as do fixed size sample procedures, but also in regard to length and pattern 
of investigation. 


It is the purpose of this paper to draw attention to some of these points and to 
indicate simple ad hoc methods by which they may, to some extent, be allowed for. 


2. The main impetus behind the development of sequential techniques has been the 
existence of the Sequential Probability Ratio Test (s.p.r.t.) procedures derived from 
the fundamental work of Wald [1]. In the original form of such procedures, two simple 
hypotheses (H, and Hy» say) are compared by the following rule, using the likelihood 


ratio Pi /Pon (of Hy to calculated from a random sample of size n:- 
) 
"Accept if < «,/(1 
Otherwise take a further opservation". 


For such a procedure we have 


(1) Pr{Accept H,_,|H,} (4 = 0,1) . 
and % are always less than 1 and usually rather small; conventional values are 
usually in the range 0.005 to 0.05. The average size of sample (a.s.n.) is 
approximately 
1- 
(2) log x, + (1- log when Hy is true. 
1- 
(3) log x, log <, when H, is true 
vhere k, = (& (log = 0, 2) 


* 221:00:000 
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An important property of the s.p.r.t., and one of its most convincing reasons for 
adoption, is that no other procedure satisfying (1) can have smaller a.s.n.'s than (2) 


or (3), when H.» H) respectively are true. 


3. Suppose that it has been decided to use a s.p.r.t. We must then, in order to 
define the rules of procedure, choose Ho» Hy and also ts % It is a remarkable fact 


that in many cases (judging from both published accounts and informal conversations) 
the choice of Hy and Hy is not primarily based on consideration of situations in which 


it is particularly important to reduce a.s.n. Perhaps as a legacy from earlier experi- 
ences of trying to obtain definitive statements of hypotheses to be tested and of re- 
quired significance level and power, there is a marked tendency to define Ho and Hy 

in terms of some physically threshold factor. Thus H_ may correspond to a (more or 
less arbitrary) specification value, and Hy to some (2180 arbitrary) limit of 
‘undesirable quality'. It may very well be quite reasonable to require certain values 
of power (or OC) for these two situations, but, also, a.s.n. minimization may well be 
more desirable in some other situation (H say) either because 


( 


a) 
or (b) experimentation is more costly when H is true (e.g. if H corresponds 
to dangerously low efficacy of a drug). 


H occurs much more often than Hy or Hy» 


Or, of course, combinations of factors of these kinds. 


4. With sufficiently detailed prior information and sufficiently powerful computing 
resources it would be possible to attain a completely satisfactory solution. Suppose, 
for example, that the physical situation is defined by a parameter 6, that 6 has a 
known prior distribution g(®@), that the average cost of sampling when a procedure S is 
used in situation @ is C(@,S) and that the average loss (due to non-optimum deci- 
sions) is L(@,S), then the average total cost (of sample and decision) using procedure 
S, could be represented as 


K(S) = face) [L(@,S) + C(@,S)] ae. 


Then the problem is to choose S to minimize K(S). Quite possibly very difficult, 
but it is a direct explicit problem. Unfortunately, the postulated wealth of informa- 
tion about g(@) and L(8,S), and sometimes even about C(6,S), is not often available. 


However, there is frequently enough information available to make possible a 
more rational selection of sequential procedure than a purely 4 prioristic choice of 
Hy» Hy combined with a conventional choice of “ss + In the following sections some 
ways in which additional information can be utilized in the construction of a procedure 
will be described. These methods are necessarily variegated, since prior information 
can exist in widely varying amount and nature. 


5. Firstly we consider a very simple case. Suppose that Hy and Hy represent the 


only possible sets of physical conditions. Suppose further that the prior probability 


of Hy is Wo» and of Hy» Wi» with W, + w, = 1. (This corresponds to the g(®) of 


section 4.) Suppose that we wish to make the (approximate) overall probability of an 
erroneous decision equal to &, i.e. 


Then, from (2) and (3), the overall a.s.n. is approximately 
1< 


as — |, 
(5) wok, log x, + (1 log + x, + log 
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In this very simple situation, with no specific costs to be considered, we can 
enunciate the problem as that of choosing Xo» “% to minimize (5), subject to (4). 


The solution is, in fact, that 7 bel should satisfy the equation 


2 2 
(l- & )(1- -xX 
(6) (x, + = 2 = | 


as well as (4). 


If k) =- ko» as is the case, for example, when two possible values of the mean of 
normal population are being compared, (6) becomes = uf (1- %) 


which on combination with (4) yields 


x) 9) 
° 1 
It is evidently desirable to have « < 1/2, a< Wo» a< w) 


Costs can be introduced by supposing, for example, that the cost of an incorrect 
decision is Cor cy when Ho» Hy is true, respectively, and the cost of sampling is 


¢ per unit, whichever hypothesis is true. *, and ol can then be chosen to minimize 


x + ( 
+c x (5) 


6. In the previous section we have seen how prior knowledge can be used to select 
appropriate error probabilities s A Now we will consider a case in which it has og 


been decided to fix oe % for values e.» 8) (> 8.) of a parameter 6, but not 


necessarily to minimize the a.s.n. at these values of 6. On the other hand, minimiza- 
tion of a.s.n. is desired at a value 6 = 6! < a ° 


Out of a number of possible approaches we select for study those in which we 
ourselves to s.p.r.t.'s between the hypotheses mee = _ and 
= 8, with error probabilities respectively, where 8) and are so 
related that Pr{ Accept 6 = remains (approximately) at the pre-assigned 
value 4: Then the question to be answered is:- what value of 8) minimizes the a.s.n. 


when @ = 


1 
Since 6 < and < will presumably be small, Pr{ Accept 6 = will be 
even smaller and the a.s.n. when @=6 is approximately 


(7) k log «,)] 


where k = [ & (log Le )) a L being the likelihood ratio of 6 = 8) to 6 = 8, 
for a single observation. ~ and 8, are linked by the equation 
' 
1 [%/(1- «,)] 
(8) = where h satisfies 
[(1- - «,)] 


| 4) 
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| 
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Ts. < 8), h is less than -1l and (8) yields 


1 
of 
1 
log (a -h log 
so that (7) is approximately equal to 
( log 


Again taking the case where 6 represents the mean of a normal population we find 


' 
h = (0, +8, - 20,)/(8, - 6) 

' ak 
k oC- @, - (6, - @) 


so that the a.s.n. at 6 = @ is approximately proportional to 


- ((6, + 


- 20,)(, + 6, - 26 
= - (0, + (3 (0, + . 


So, within the accuracy of this approximation the a.s.n. at @ = 6 is minimized when 


i 
(8, + 8) = 5 (6 + 
e =e, -@ 
The ratio 
a.S.n. using s.p.r.t. with 8) 8) ) 
" n n 8) = 
is approximately | 
8, - 6 -6 


7. #$XIf we are interested in reducing a.s.n. for values of 6 between 8, and 8 other 


techniques, of recent development, are available. This problem has received special 
attention, perhaps because it is in this region that the a.s.n. of the s.p.r.t. reaches 
a maximum. It is known, for example, that in the normal case discussed in the previous 
sections, the s.p.r.t. with «, = % < 0.008 has a greater a.s.n. when (1/8, + 8,) 
than the size of the fixed size sample needed to discriminate between the hypotheses 
6= Pe 6= 8 with the same power. 


The techniques developed to deal with this type of probist actually change the 
form of test from the standard s.p.r.t. procedure. Anderson 2) has described a 


promising procedure which is essentially based on the comparison of A with certain 


linear functions of n in place of constants, as in the s.p.r.t. By trial and error 
he was able to obtain a test with a.s.n. at 6 =(/Xe, + 8,) not much in excess of a 


lower bound for the a.s.n. of any test with given discriminating power (between 


MET 
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6 = and 6 = 8,) obtained by Hoeffding!), The table below, based on information 
in [2] and [3] illustrates the properties of Andersons's test. 


a.s.n. for 


wee Fixed size Anderson's Lower 
° 1 sample _ S.p.r.t. test bound 
0.01 30, +6,) 541.2 527.9 402.2 388.3 

°. or 8) 541.2 225.2 249.4 225.2 
0.05 +6@,) 270.6 216.7 192.2 187.0 
8, or 6, 270.6 132.5 139.2 132.5 


Professor W. J. Hall, of the University of North Carolina, has kindly informed 
me of a method which promises to provide useful test procedures while avoiding the 
necessity for such heavy calculations as we entailed in constructing optimum proce- 
dures by Anderson's method. He proposes to replace the standard s.p.r.t. based on 


= P/Pon by two 8.p.r.t.'s, one between 6 = and 6 = 
the other between 6 = e and @ = 8, (where e' is the value of 6 at which aiesieatiadhnies 
of a.s.n. is desired, and < The procedure terminates when @ = is 
accepted by the first ey or 6 = 8) a the second test, but not when 6 = 8, is 
accepted by either test. It seems reasonable to take error probabilities %» “, when 
6= 8» 8 = 8 respectively, and, perhaps, to base the error probabilities when @ = 


on the value of the OC at 6 = 6 for the standard s.p.r.t. discriminating between 
6= with error probabilities respectively. The selection of error 


probabilities does not yet seem to be well-established. But this does seem a practi- 
cally useful way of constructing a test procedure appropriate to the conditions 
described. In the normal case it gives a procedure of the same form es Anderson's. 
It is, of course, quite a general method, not limited to the normal case. 


8. This paper cannot be more than a rather superficial survey, because it describes 

a relatively unexplored field with many possible, and practically important, variations 
in detail. But the cases described should suffice to show that variation on standard 
sS.p.r.t. techniques, to meet special requirements, need not call for very deep or heavy 
mathemetical work, while it should also be clear that it is important, even essential, 
to allow for these special requirements. There should be constant care not to become 
hypnotized by the two hypotheses and two error probabilities which play a central role 
in formal sequential analysis. 


Some problems of the kind described in section 6 have been treated in a more 


detailed manner in an earlier paper by nyseir!4] , The present paper is, however, much 
broader in scope than [4], which also contains some numerical tables of use in con- 
structing sequential procedures. 


REFERENCES 
[1] Wald, A. Sequential Analysis, New York: Wiley (1947) 


[2] Anderson, T. W. “A modificatior of the sequential probability ratio test to 
reduce the sample size", Ann. Math. Statist., Vol. 31 (1960), pp. 165-97. 


[3] Hoeffding, W. "Lower bounds for the expected sample size and the average risk of 
a sequential procedure", Ann. Math, Statist., Vol. 31 (1960), pp. 352-68. 


[4] Johnson, N. L. "On the choice of a sequential procedure" Proc. Biometric Society 
Symp. on Quant. Methods in Pharmacology, Leiden (May, 1960). 
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Fred Ennerson, Jr. and Mark Manning 
Hughes Aircraft Company, Tucson, Arizona 


A common method used for analyzing test results of complex units is to graph unit 
performance based on review of the test data. This can be done in two ways -- analysis 
of individual test measurements, or monitoring quality levels on an attribute basis. 


Any one test may involve a considerable number of test steps, each of which might 
be subject to detailed analysis. Regardless of benefits obtained in monitoring indi- 
vidual test measurements on the floor with testers plotting X and R-chart data, com- 
plex unit testing may involve dozens of charts for any one test position. To keep the 
program in operation, this would present a major problem in the piling up of paper- 
work. Hence, the "go, no-go" approach permits quality level floor controls without 
time-consuming clerical work. 


The authors have devised a method whereby the individual tester can monitor 
quality levels without waiting for performance interpretation at a later time. This 
is handled on a single “Test Performance Chart" (see Fig. 1). 


These test performance charts are a modification of those copywrited in 1955 by 
Emerson Research Laboratories. They are posted on each test position where they are 
clearly visible and easily plotted. The results of each test are noted immediately 
after the complete test cycle. 


This chart is made up of a series of grids, as illustrated in Fig. 1. Testers 
plot only initial tests, which gives the true manufacturing quality level at all times. 
If the unit is completely acceptable, the tester marks an "X" on the diagonal “right- 
upwards". If a unit fails for any reason, the tester indicates a "right-downwards" 
plot on the diagonal. To provide supervision with information concerning differences 
between testers on the same position, or differences for two-shift operation, different 
symbols may be used to indicate unit acceptance or suspension. 


Each chart is originally entered adjacent to the chart apex with an appropriate 
plot indicating test acceptance or suspension. Subsequent plots are entered until the 
entire plotting area is used up. The chart is basically designed for 100 tests. 


The slanting guide lines indicate acceptance levels in terms of accumulative 
acceptance rates. As soon as the tester plots the most recently completed test result, 
he has before him, and to anyone else within the area, the exact accumulative test 
acceptance rate for the indicated number of units tested. Where several test posi- 
tions are used to test the same unit, these test performance charts are very effective 
in determining if acceptance rates are compatible among the test positions. Similarly, 
these charts readily show which units of various types are going to require additional 
engineering effort to improve the performance of any one unit. 


At the completion of each test performance chart, the tester obtains a new chart, 
fills in all printed headings, and again enters test results beginning adjacent to the 
chart apex. After two such charts are completed, they are analyzed to determine what 
the authors choose to call a “control cone." These control cones, essentially 2-sigma 
control limits for "p" charts, are marked as indicated in Fig. 1. At the completion of 
the third and subsequent charts, if the last plot on each sheet falls below this con- 
trol cone, the tester notifies his immediate supervisors promptly, and investigation is 
made to determine the cause of this falling off in process quality. It is, of course, 
just as important to find out why the process improved if the last plot per sheet 
falls above the control cone. As supplementary information, testers also indicate on 
the chart information concerning product acceptance changes, such as test position 
calibration, engineering changes, new manufacturing lot initiation, test position 
modification, etc. Subscripts are noted on the chart to indicate such changes, with 
brief remarks entered near the chart apex. 


The test performance charts are also used to determine the effectiveness of an 
engineering change. For example, subscript #3 as shown in Fig. 1 indicates the in- 
corporation of an engineering change. It is evident that product quality quickly 
improved. It is possible to calculate how many units are required to detect such an 
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improvement and how many suspended items are allowed for this improvement in product 
quality. 


Further statistical information is furnished to the tester, designated as a 
“failure run limit" and a “failure rate." The failure run limit is provided to permit 
the tester to know exactly when successive failures are indicating a significant de- 
terioration of test quality. 


Let us assume that we have a test position which has shown an average acceptance 
rate of 80 percent over 200 tests (2 sheets). The failure rate is thus 20 percent. 
This 20 percent failure rate is the same as 1:5. Therefore, the probability of finding 
a significant deterioration in test quality may be determined. At the observed fail- 
ure rate of 1:5, the probability of testing two discrepant units successively would be: 


1 


(1/5) (1/5) = 25 
Obtaining three discrepant units successively would be: 


1 
(2/5) (2/5) 


The odds for noting three discrepant units are seen to be very small (1:125). ‘The 
authors consider odds of 1:100 as significant for this type of operation. Consequently 
with the example given, the failure run limit would be given as three (3). The fail- 
ure rate would be given as 1:5. 


This failure rate can be used to substantiate product quality deterioration. If 
the failure run limit is reached but there is no apparent cause or pattern of test 
failure, an additional number of tests may be made as prescribed by the failure rate 
to try to detect the cause. For the above example, five more tests could be run. If 
only one failure was observed, testing would be permitted to continue. 


This concept of seemingly giving more leeway in terms of "take another look” is 
a@ conciliation between setting down hard and fast statistical ground rules and the 
problems of maintaining schedule and supervision's cooperation. 


Once the control information is provided, regular reviews of subsequent sheets 
are maintained by the quality engineer with a view toward upgrading control cones and 
appropriate changes in failure run limits and rates if significant product quality 
improvement is noted. 


The test performance charts require a minimum amount of the testers' time to 
maintain. These charts are always up to the minute, indicating the present quality 
level of the product going through a test position. The testers, through channels, 
are able to inform quality and engineering personnel when the failure run limit has 
been reached for a particular test position so that prompt action can be taken to 
prevent excessive failures. On the completion of each sheet, the tester is also able 
to inform cognizant personnel when there has been a significant change in the process 
average. Wo clerical or analytical work is required from the testers to maintain these 
charts. 


For purposes of maintaining long range quality histories of a process, a "p" chart 
(not shown in Fig. 1) is attached to the lower left-hand corner of the test performance 
chart holding fixture. On this "p" chart, the accumulative test acceptance rate is 
plotted at the completion of each sheet. Control limits are provided to indicate the 
normal expected variability. 


The success of this program was based upon the following: 


(1) Simplicity of operation 

2) The desire of quality and engineering personnel to be part of this program. 

3) The placing of direct responsibility upon the tester to initiate corrective 
action for "on-the-spot" feedback of information to responsible departments. 


| | 


References: 
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Some of the results obtained in this program are: 


1) Improvement of product reliability 

2) Reducing rework and scrap 

(3) Promote operator and/or inspector interest on the job 
(4) A guide for production supervision 

(5) Am effective tool for engineering. 


(1) Morten Gale "Plotting of Success or Failure Test Results" 
Emerson Research Laboratories 
(2) Published in Industrial Quality Control magazine September 1959. 
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INDUSTRIAL APPLICATIONS OF NONPARAMETRIC STATISTICS 


Edwin C. Harrington, Jr. 
MONSANTO CHEMICAL COMPANY 


Very often, in industrial and in scientific investigations, we are 
faced with the problem of analyzing and interpreting data obtained from 
an arbitrary and subjective scale of measurement. The appearance of a 
painted object after exposure to the weather might be rated as poor, 
fair, good, or excellent. Or we might have several batches of one 
product, of which one can only say that batch A is better than B, B is 
better than C, etc. In situations of this sort, although we can 
attach numbers to the several classes, the numbers indicate order, 
only - i.e., an ordinal scale - and the usual properties of numbers - 
addition, subtraction, multiplication, and division - are no longer 
valid operations. For example, batch D, ranked as No. 5, is not 
necessarily three “units" worse than batch B, ranked as No, 2, 


For this reason, the common methods of analysis based on an interval 
or ratio scale of measurement, do not apply. Many of the well-known 
statistical methods, "t" tests, the analysis of variance, even X and 

R control charts, depend upon the numerical properties of an interval 
scale, i.e., a scale in which the interval between ranks has meaning. 
Another restriction to the use of the common analytical procedures is 
their assumption an approximately normal (Gaussian) distribution of 
the observations on this scale, To meet these difficulties, the so- 
called distribution-free nonparametric procedures have been developed, 


Consider the problem of comparing two paints for weather resistance. 
Ten panels might be prepared from paint A and ten from paint B. All 
twenty are exposed to the weather for a year and then compared for 
overall appearance. Quite a number of measurements might be made: 
total exposed area, number of pinholes, total blisterec area, length 
of checks and cracks - all of questionable reliability and probably 
none of greater practical value than a simple ordering of the twenty 
panels from best to worse. 


If the A's are all better than the B's, or vice versa, there is 
hardly any question of interpretation. The recording from good to 
bad would be: 
AAAAAAAAAABBBBBBBBBB 
and the conclusion obvious, 
The other extreme situation would result if the two paints were really 
the same, Here we would not be favored with a nice symmetrical 
pattern, such as 
but rather, with a random distribution of A's and B's, such as: 


ABABBBAABAABABBBAABA 
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If the expected pattern were symmetrical ABABAB, it would be easy to 
detect even the smallest difference between A and B. Since the expected 
pattern is actually random, it becomes more difficult to decide that a 
difference does exist - and also, a larger difference must exist 
between A and B if one is to detect it. 


Suppose in a test of the two paints, A and B, the ordering from good 
to bad were as follows: 


AABAAABABABAABBBABBB 


The A's trend toward the good end and the B's toward the poor end, and 
yet it does not seem too unlikely that this distribution could arise by 
chance (consider the tossing of a coin). The statistical problem is to 
calculate the probability that this distribution could actually arise by 
chance, If the distribution could reasonably be accounted for by chance, 
there is not sufficient evidence to say that the two paints are 
different. On the other hand, if it is relatively unlikely that this 

is a pure chance distribution (say it would occur by chance less than 

5% of the time), then we can reasonably say that B is poorer than A. 


The Mann-Whitney U Test. 


The non-parametric procedures, of which there are now very many, are 
designed to calculate these probabilities. One of the most useful and 
most efficient of the nonparametrical procedures is the Mann-Whitney U 
Test, provided the conditions for its application are met. This test 
applies where two independent groups are compared, and where they are 
arranged in an ordinal scale, such as from large to small, from good to 
bad, from smooth to rough, etc. The Mann-Whitney U Test is appropriate 
in this paint film example. 


The procedure is to calculate the statistic U, and it is done in 
this manner: 


1. Arrange the observations in a single ordered series, 
from best to poorest, from most to least, or by 
whatever criterion is appropriate. 


2. Of the two independent groups, identify one as the ny 
group (usually the smaller), and the other as the 
No group. 


3. Starting at either end of the ordered series, count the 
number of times a member of no precedes each member of ni. 
The sum of these counts is U, or its compliment, U'. 

(U' is what you would get if you started to count from the 
wrong end. It is related to U by the equation: 


U = mn) - Ut 


Intuitively, it is not difficult to see how U varies with the type of 
ordering within any series. If all of the A's precede all of the B's, 
U is obviously zero. As the A's and B's become more intermingled, the 
value of U will increase from zero to some maximum value, approximating 
njn,/2, which will occur when the A's and B's are alternate. From the 


point of view of hoping to distinguish between the two groups, we 

would normally hope for a small value of U. As U increases, there is a 
critical value, beyond which it must be concluded that the distribution 
of A's and B's is due to chance, and these critical values are 
published (1) for various values of n, and n,. For ten observations in 
each group, the critical value (Pr=0.05, two“ tailed) is 23. 
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For the ordered series of the paint exposure test, there are ten 
panels in each group, so we may arbitrarily select the A's as the n, 
group. For the first A, there is no preceding B, and the count is 
zero. Similarly, the next count is zero. However for the third, 
fourth, and fifth A, the count is one for each A. This counting 
process is continued for all A's, and the total U score is 


U = 


Since the calculated U is just at the critical value, this means that 
there is only a 5% probability that the pattern of A's and B's could 
result from chance of two identical paints, and so it is concluded 
with 95% confidence that A is really superior to B, 


Because the panels were not graded against a standard scale, it is not 
possible to express their merit except in relation to each other, 

This relation is correctly expressed in terms of the median (averaging 
is not allowable on an ordinal scale), and the median rank of A is 
7.0, and that of B, 14.5. 


The non-parametric procedures are useful not only in situations where 
their use is required, but also as optional procedures where the 
standard parametric methods would ordinarily be used. In this 
application there is usually a considerable saving in time with some, 
but often negligible, loss of efficiency. 


Had we measured the surface gloss of the paint panels with a suitable 
gloss meter, data would be available for analysis by means of the 
parametric "t" test. This would require a moderate amount of 
calculation, and would require that the observations be normally 
distributed. The Mann Whitney Test could se completed by inspection, 
and it requires no assumption of normality. 


The Binomial and Sign Tests. 


At the present, there are many non-parametric methods available, of 
which the Mann Whitney Test is typical. Some of these have immediate 
and useful application in the interpretation of sontrol charts, as, 
for example, the Binomial Test of which the well-known Rule of 7 is 

a special case. If observations are equally likely to fall into one 
of two classes, as the points of a control chart are equally likely to 
fall above or below the median, then the probability of a pattern such 
as 10 above and 3 below is readily calculated by the binomial 
distribution. The question a production man would ask is, does this 
pattern of 10 above and 3 below indicate a change of level? Should 
corrective action be taken? Calculation, or reference to a table, 
will show that such a pattern will occur about 9% at the time even 
when no change in level has occurred. And the recommendation would 

be to wait for more convincing evidence before changing process 
controls (thus avoiding overcontrol) unless defective material is 
actually being produced. The Binomial Test is quick and easy, 
requiring only the ability to count and the availability of a table 


The Sign Test is an extension of the Binomial, and it is useful in 
comparing two materials when they are tested in pairs. Consider the 
evaluation of two glues, X and Y. The major problem in evaluating 

glue is to get a uniform substrate, particularly if this is wood. 

It is accepted procedure to divide each test piece, and treat one half 
with X and the other with Y. Each half is then split, glued, and 
broken. Based on the appearance of the break (failure through the glue 
line or failure through the wood), X is rated superior to Y, or Y 
superior to X, Quantitative measures of the type of break, which are 
sometimes used, are of questionable value and are very often misleading. 
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In the sign test, a plus sign is used to indicate the superiority of 
one sample of the pair, and a minus sign to indicate the superiority 
of the other sample. Judgment of whether there is sufficient evidence 
to conclude the overall superiority of one sample or the other is 
based on the distribution of the plusses and minuses by comparison, 
again, with tables of the binomial distribution. Thus, if 100 pairs 
are tested, a split more divergent than 40:60 is sufficient evidence 
to conclude with 95% confidence that one sample is superior to the 
other. 


In this example, not only is the sign test much easier to apply, it is 
also preferred to a "t" test based on measures of percent "wood 
failure" since the latter are far from normally distributed. The wood 
failure distribution is markedly "J" shaped, ané thus the assumption 
of normality is not obtained. 


A discussion of non-parametric statistics could well occupy a full 
semester course, for the ingenuity of many statisticians has 
developed an extensive field. Non-parametric Statistics, by S.Siegel 
(McGraw-Hill) is an interesting and extremely valuable discussion of 
the theory and methods currently available. 


In summary 


1. Where data are non-parametric to start with (scores, 
ranks, grades), the analysis should also be based on 
non-parametric procedures. Care should be taken to 
avoid a not uncommon practice of substituting numbers 
which are then treated as parametric. 


2. Where data are parametric, (measured on at least an 
interval scale), it is occasionally true that the 
assumptions of parametric analysis are not obtained 
(in particular, the assumption of near-normality). 
In this case, a non-parametric procedure should be 
used in the analysis of the data, 


3. When parametric analysis is appropriate, the simplicity ' 
{ of the non-parametric methods may still recommend their 
: use. There is always some loss of efficiency, but this 
is important only in those borderline cases where a 
7 detailed study is then most certainly warranted. 
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KEEP IT ON THE SQUARE 


R. J. DeGray 
The Standard Oil Company (Ohio) 


The life of a quality control man is pleasant, so long as all of the little dots 
stay cozily within The Control Limits. But when the dots start “headin' for the 
Border", the situation changes. What does the QC man do then? There are at least 
three possible courses of action. 


a. Press the “out of control" button, and let Production clean up the mess. The 
@C'er who is content to spend his life separating the product into piles marked "Good" 
and "Bad" will take this course. 


b. I'm sure cuss has been said better by somebody else sometime ago, but appropi- 
ate references have eluded me. If the Quality Control group has established the proper 
relationship with the Production Group, QC and Production will roll up their sleeves 
and, together, find out what's wrong and how to fix it. 


c. If the situation is really idyllic, QC and Production would have studied the 
process together, some time ago, and would know which lever to pull or which valve to 
turn - and by how much - before such knowledge is needed. 


Alternates (b) and, preferably, (c) require that the Quality Control people study 
the process as well as the product. Now I can cite references, sticking to IQ. 
Harold Kellogg (1) put it this way: 


Statistical With the help of numbers or data, 
Quality we can study the characteristics of any process in order 
Control to make it behave the way we want it to behave. 


This importance of process study has long been recognized by the American Society 
for Quality Control, which, through the Chemical Division, sponsors courses, confer- 
ences and symposia on Response Surfaces and EVOP (2). That production experts and 
statisticians should work together is pointed out by Bicking (3). A subsequent article 
by Bicking (4) separates the process study group from the quality assessment group, but 
this can be afforded only by the larger companies. In general, a Quality Control engi- 
neer should be able and willing, to study processes as well as products. This paper 
discusses one simple little rule which will help us to plan our study so that we obtain 
more information with less work. That rule is - KEEP IT ON THE SQUARE. 


To illustrate, let's look at something that's not on the square. Figure 1 is a 
photograph of a laboratory scale of common type. It has three feet, two of which are 
adjustable in height. On the base is a little spirit level of the bull's-eye type. 

To level this balance, you screw the two adjustable feet up or down, as needed. But 
it's not simple. You screw one foot up or down until the bubble is close to the bull's 
eye, then screw the other foot to home in. But changing the second foot throws off the 
adjustment made with the first foot, so you have to diddle back and forth. In mathe- 
matical language that's successive approximations. It's tedious. 


Figure 2 shows what's wrong. This is a plan of this balance. As Foot A is 
screwed up or down, the base rotates on the axis BC. As Foot B is screwed up or down, 
the base rotates on the axis AC. But angle ACB is acute, so motion of the bubble 
toward BC, has a component toward AC, and vice versa. 


Figure 3 is a photograph of an analytical balance. This type, or something more 
modern, is well-known to all of us. It, too, has two adjustable and one fixed leg, but 
levelling it is easy. Figure 4 shows why. Again, this is a plan view. P and Q are 
two spirit levels of the tubular instead of the bull's-eye type. This is a convenience, 
and helps in explanation, but it's not necessary. The vital point is that angle ACB is 
a right angle, or 90°. Now when Foot A is screwed up or down, the base rotates around 
axis BC, and the bubble in Level P moves back and forth, but the bubble in Q merely 
rolls around in the tube, for Q is perpendicular to AC and parallel to BC. Foot A can 
be adjusted until the bubble in P is centered. Then Foot B can be adjusted until the 
bubble in Q is centered - and the balance is level. The P bubble won't budge as the Q 
bubble is shifted, and vice versa. The secret of success is 90° - this set-up is On 
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the Square. This is the difference between a tedious, cut-and-try procedure and a 
simple, direct and unequivocal technique. 


There's nothing mysterious about this. Whenever you can get two forces - two 
vectors - two variables - two factors to move at right angles to each other, their ef- 
fects will be mutualiy independent. If Foot A is Temperature, and if Foot B is pres- 
sure, and we keep these at right angles to each other, we can move Temperature up and 
down and see what this does to the process, and we can move Pressure up and down, and 
see what this does to the process, and each effect will be independent of the other. 
We'll get two direct, dependable, unequivocal answers: The effect of Temperature, and 
the effect of Pressure. If our experiment is not On the Square, we'll get an answer 
which is partly due to Temperature, partly due to Pressure, and we'll never be sure 
which is which. 


What's this about Temperature being perpendicular to pressure? If you use a 
dimensionless scale, this is more easily accomplished than explained. 


If everything is perpendicular to everything else, the angles are all 90°, so the 
angles are all equal. The Greeks, of course, had words for such situations ~ ortho is 
equal, gonal is angle, and orthogonal is equi-angled. If one variable is orthogonal to 
another, the vectors are perpendicular to each other. Keep the variables orthogonal, 
and the experiment will be On the Square. 


How do you recognize orthogonal variables when you meet them? The variance of 
their cross-products will be Zero. Then everything is independent of everything else, 
’ and the experiment is simple to run, simple to calculate, and definite in its answers. 
You can “level-out" the process in a few simple stages. 


Let's take a simple illustration. We want to find out what happens to tensile 
strength when we (a) shift the temperature from 500° to 600°, (b) shift the reaction 
time from 4 hours to 8 hours and (c) use 5% or 10% of Supposium Perhapsonate. 


First, let's do some Orthogonal Coding. Three simple equations can be set up - 


A = Temperature - 550° 


Time - 6 hours 5 
= 
Hours 


SP - 7.5% 
2.5% 
This simple trick means that when we use a temperature of 500°, we simply say that 
A is -59 over 50, or -1. When we use a temperature of 600°, A is +l. Similarly, B is 
either -1 or +1, and C is either -1 or +1. That's sort of handy, isn't it? 


Now we'll set up some experiments: 


Run Temp. Time S.P. A B c 
1 500 + 5 -1 -1 -1 
2 600 + 5 +1 -1 
3 500 8 5 -1 +1 -1 
+ 500 + 10 1 -1 +1 


You'll recognize this as the ordinary approach - start with the standard state and 
bump each @ariable at a time. What do we get out of this? Try the cross-product of A 
and B - it adds to 0. But the sum of A is -2, and the sum of B is -2, and -2 times -2 
over 4 is +1, so the variance of AB is 0 - (+1) or -1. It's not zero; we can't evalu- 
ate A independently of B. AC and BC are in a similar fix. 


How do we evaluate A independently of B and C? By using Runs 1 and 2 alone. i 
AB 0-0-0 


ac 


BC +2 - (+2) =0 
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To get the most information for the least effort, we'd like to 
to answer all three questions - what's the effect of A, or B, or of 
to use only Runs 1 and 2, Runs 1 and 3, and Runs 1 and 4. Only Run 
milked of its full value. How can we do better? By changing Run 1 
to all high values. 


Run Temp. Time S.P. A 
1 600 8 10 +1 
2 600 + 5 +1 
3 500 8 5 ol 
500 10 -1 

0 


yet we did no more work. 


to these designs are Cochran and Cox (5) and Chew (6). 


or the whole business is null and void. 


placed our thought processes with a bunch of stupid electrons. 
Here's what's going on. We say: 


= bo + + box, + bax, + b.x 


The next step in this foolishness is to run a t-test on each "b", to find the one 
that is least significant. This one is discarded, and the (n-1) x's are put back into 
the Brain. New answers are spewed out. Which set of answers is right? 
another t-test says that we should discard another, and another, and another. 

i time around, the surviving "b's" shift. When you finally stop this silly business, you 
: report the "b's" as Gospel. But you know perfectly well that one more round would give 
a new set of answers. Aren't there any definite answers in this business? 
less the "x's" chosen are orthogonal to each other. And then you don't need a Giant 


Brain at all. 


The same is true for B, using Runs 1 and 3, and for C, using Runs 1 and 4. 
doesn't tell us anything new - anybody can see that moving A up and down while holding 
B and C constant will give you a measure of A independent of B and C. 
to show that we get the same sensible answer from the orthogonal stand-point. 


use all four runs 
It's wasteful 
1 is really being 
from all low values 


Cc? 


All of this 


All I've done is 


c 
+1 
-1 
-1 
+1 
0 


All these variables are orthogonal to each other - perpendicular to each other - inde- 
pendent of each other - On the Square. By the simple trick of switching from all low 
to all high, we make it possible to use all four runs for all three answers. 
ing on the Square (or cube, in this case), we increased the power of our work 41.4% - 


By Keep- 


You recognize this design as a half-replicate of a 3-factor, 2 level factorial. 
All 2-level factorials are orthogonal. Setting up a 2” factorial is a simple, cook- 
book operation, but it also is an infallible method of drawing 63 lines all perpen- 
dicular to each other (in 63 dimensional space, of course). 3" factorials are ortho- 
gonal for all quadratic terms. The Plackett and Burman designs are orthogonal. 
Box-Wilson Center Composite, and the Box-Hunter Rotatable Designs are orthogonal. 
DeBaun method of blocking these designs does not upset their orthogonality. 
Balance is not, was not, and never will be orthogonal. Two of many possible references 


Random 


Keeping on the Square adds power to our work. In the case of Analysis of Vari- 
ance, however, orthogonality is more than a convenience; it is an absolute necessity. 
I Every single term in an Analysis of Variance must be orthogonal to every other term, 


Finally, I'd like to touch on Multiple Regression. Back on the old and not so 
good days when all we had was a hand-cranked calculator, we thought and planned before 
inverting a matrix. Now many of us take any old mess of data and pour it into the 
hungry maw of a Giant Brain and believe implicitly whatever spews out of the other end. 
It's all so quick and easy and untouched by human hand - or brain - that we have re- 


How many x's do we use? The tendency seems to be to drag in all we can think of, and 
then one more as an offering to the Unknown God. The Giant Brain doesn't know or care 
whether our choice of variables makes sense or not; for every "x" that we put in, the 
machine will give back a "b". And if we have 50 data points, put in any old 49 x's 
that we please, and the machine will give a perfect fit. Serves us right. 


Neither, for 
Every 


No -- un- 


The 


4 


421 
4 
B 
+1 

+1 
1 
0 


422 1961 ASQC CONVENTION TRANSACTIONS 


All of the data and the "x's" - fair or foul - are reduced first to Normal 
Equations. Then the normal matrix is inverted, and the "b's" come tumbling out. The 
Normal Equations look like this. 


by Sx,2 + bySx)x9 + + b,Sx)x, +------ = 
by Sx 1x2 + b2Sx22 + b3Sx2x3 + bySxox, + ---- = Sx2Y 


And so on and on. Most machine programs take the raw data, set up the Normal Equations 
solve these equations, and give you the answers without bothering you with these de- 
tails. But, if you could see inside one of these monsters, you'd spot one Square term 
and (n-1) cross-product terms on each line. If your data are orthogonal, the cross- 
products all equal zero. There's nothing left in here but us squares. To get bj, 
merely divide Sx,Y by Sx,2, and so on - a desk calculator job. 


What is more important is that, if b,)5 is practically zero, throw it out. Don't 
recalculate, for not another "b" has turned a hair. The x's are orthogonal - the 
variables are independent of each other - the equations are On the Square, and the 
answers are unique - definite - unaffected by the addition or dropping of other 
variables. 


To be practical, we must recognize that there are many situations where all of 
our variables are not subject to our control. Sometimes the shift foreman influences 
the results, and people are notoriously non-orthogonal. Without air conditioning, we 
have to take humidity as it comes. However, it is still true that every variable that 
can be handled orthogonally should be ndled orthogonally. A 30 x 30 matrix in which 
28 factors are orthogonal and two are incorrigible can still be handled easily on a 
desk calculator, and the elimination of non-significant variables will have a small 
effect on the other regression coefficients. The square can be warped a little without 
losing all of its virtues. 


To summarize: By using a little foresight in planning our work, we can choose a 
design which will arrange the variables orthogonally. This will: 


1. Give us more information for less work. 
2. Permit a valid Analysis of Variance. 
3. Give definite answers with a minimum of calculating effort. 


KEEP IT ON THE SQUARE 
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FIGURE 3 
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SHORT CUT STATISTICAL METHODS 
as applied to certain 
Reliability problems 


Gordon H. Reckhart 
Radio Corporation of America 


INTRODUCTION 


As these methods may seem to by-pass a great deal of statistical theory generally 
applied to reliability problems, it would be best if first a little time was spent des- 
cribing the particular problem confronting us. Our product line consists of a complex 
mix of heavy electronic equipment including many non-defense systems used for physics 
research. These equipments are generally needed before we have time to design and 
build them. Due to the time schedules and economics involved, selection methods must 
be streamlined. This paper describes some of the methods used and under what circum- 
stances they are applicable. 


RELIABILITY MATRIX 


We believe that reliability may only be achieved by coordinated actios of all 
groups, from design to final emplacement of the equipment. In particular, Design, 
Production, and Field activities greatly influence the final reliability. In the de- 
sign area, you will note, there are many activities that effect design reliability and 
a few are listed below as factors in the reliability matrix. 


EQUIPMENT AVAILABILITY 
DEPENDS ON COORDINATED 


CONTROL OF 

vif 
DESIGN FIELD 

PRODUCTION = 


Each of these reliability factors is an engineering discipline requiring special- 
ized professional knowledge and experience. Omission of any factor will eventuw lly 
result in system shortcomings, while over emphasis on any one factor will increase the 
costs without necessarily improving system worth. Maximizing system worth per dollar 
expended is a managerial task requiring careful coordination and balance of the techni- 
cal factors listed on the reliability matrix. 


Part selection is one of these technical factors and judicious selection of parts 


will assist in the prevention of excessive assembly, renair and maintenance costs of 
the equirment. 
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WHAT IS RELIARILITY 


Most published definitions of reliability make statements similar to "Reliability 
is the probability of performing without failure a specified function under given con- 
> 


ditions for a specified period of time." 


This statement has been agreed uvon by the best minds as being the most accurate 
and inclusive. Unfortunately, the psychological effect has been one of mass hyonosis 
within the electronics industry. The reason for the mass hypnosis is that the unso- 
phisticated are lulled into a false sense of security that they know something about 
the reliability of parts if they have statistical data. Actually a part cannot be con- 
sidered reliable until every failure mode is known, understood, measured, and controll- 
ed. If one has this knowledge, one may design around weaknesses with sufficient safety 
margin to prevent failures or to minimize the effect of the failure in the expected 
operating environment. 


ACTIVITIES SUPPORTING 
ATTAINMENT OF GO&LS 


DESIGN FIELD 


PRODUCTION 


RELIABILITY SUPPORT MATRIX 


While the do-it activities hold the option for ultimate equipment reliability 
they receive strong support from service activities. You will note that design reviews 
are included in the support matrix. The fact reviews are part of the supnort matrix 
does not imply that design reviews are organizationally separated from design but 
merely that this function is available within the organization to assist the designer. 
Several activities are listed on the supvort matrix to which statistically trained 
engineers are likely to be assigned. In these supporting activities the statistician 
is quickly introduced to all aspects of reliability enhancement. 


TIME PHASING TYPICAL R & D PROGRAM 


It is apparent from this schedule that if we have a contract calling for delivery 
of equipment two years hence, we will vrobably have a great difficulty meeting our 
schedule if a ten-thousand-hour life test is performed before parts are chosen for in- 
corporation with the design. In fact, the life test for parts would probably use up 
most of the design and production time if we delayed the choice of parts until the con- 
clusion of the life test. Let us assume for a moment that we have decided to run 
longevity tests on parts and examine a tynical flow diagram of life testing activities. 


A 
. 
? 
| RELIABILITY SUPPORT MATRIX 
f 
Sa ™ | 
“te, BE? 


HOW WILL LONGEVITY TESTING 
ASSIST THIS PROGRAM? 


SYSTEM CEVELOPWENT { 


MAROWARE SELECTION 


SYSTEM CHECKOUT 


Looking at this diagram, one sees that if all goes well in the life test, there 
will be at least one cycle of ten-thousand hours. However, if the test is not success- 
ful, there may be many more ten-thousand hour cycles. In addition, by the time several 
life test cycles have been completed the equipment may be obsolete; a new part may 
have been develoved which will be far more satisfactory from a performance point of 
view than the original part; or major circuit changes may have been added which elim- 
inate the need for the pert. Therefore, on our ground equipment which is amenable to 
check-out and maintenance procedures the effects of part failure may often be avoided 
or side=stepped if it does not arnear advisable to engage in considerable time, effort, 
and expense in longevity testing. Naturally those parts whose procurement will embrace 
several systems development cycles are subjected to longevity tests unless field in- 
formation is already available. 
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AN ALTERNATIVE PROCEDURE 


This procedure is based on knowing, understanding, measuring and controlling 
failure modes. Therefore, we first study possible failure modes and choose the most 
rugged part or parts with respect to the mode of failure. Should weakness be revealed 
we try to increase the parts resistance to stress. Then our design checkout and main- 
tenance procedures are adjusted to minimize the effect of failures upon the performance 
of the equipment. 


Should we desire to determine nert longevity we have the operating time of the 
equipment which will accumulste part-hours far faster than any life test stand. If 
you sre not interested in determining mean-time between failure and keeping the neces- 
sary records but are interested in finding the best part, there are statistical techni- 
ques available for sampling the part distribution at intervals. One such system is 
based on starting with a 50-50 split of parts between suppliers. 


Whenever a part fails it is replaced with a part made by the other vendor. Even- 
tually the distribution of parts between vendors will indicate which part lasts the 
longest. 


WHAT IS BEST ABOUT A PART RELIABILITY-WISE? 


The answer to the question as to what is best about a part fram a reliability 
point of view is a function of operational mission of the equipment, its environment 
and the maintainability of the equipment. In many applications, low failure rate is 
the most desirable property. However, the ability to withstand overload, a predict- 
able wear-out period, or built-in indicators of malfunction may be even more desirable. 
Hence, to use a short-cut statistical method, one must first determine what attributes 
of a part are desirable and then conduct tests which will determine with a given risk 
that the best source or supplier of the parts has been selected. If we are selecting 
parts for equipment which may be designed for ease of maintenance, it appears that we 
should compare the gross stress withstanding ability of parts or the distritution of 
stress withstanding ability of parts. Our final choice should be that part most resis- 
tant to the various failure mechanisms. 


STATISTICAL METHODS 


This section will discuss six statistical methods; two of which are concerned 
with selecting the best based on percent defective, two of which are concerned with 
comparisons of percent defective, and two of which are concerned with the representa- 
tiveness of samples and the selection of a population with the best median. The latter 
two use variables data. 


2 
Procedure I 


This procedure guarantees a probability P* of a correct decision regardless of the 
true configuration of the unknown parameter values (fraction effective). 


To use this procedure in Table I you must: 

1. Decide on the percent difference d* you wish to detect between the best and 
next best povulation. 

2. Decide on the probability level P* that you wish to guarantee for selecting 
the best when the true difference between the best and next best is at 
least d*. 

3. Look in the Table for the sample size "n" that must be tested fran each 
population. 


Example: We wish to be 80% sure of picking the best source when the best is at least 
20% better than the next best. If we are comparing sources, the required sample 
size is 23 items to be tested from each source. If we wish to be 99% sure of picking 
the best source in the same problem the required sample size is 89 items from each 
source. 


Since these parts are being tested at a judiciously chosen stress level, the 
percent defective of the best parts may be relatively high and difference between it 
and the next best relatively lerge. 


Ny 
J 
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2 
Procedure II 


This is a quick method of sepvaratine several sources, populations, or vendors 
into two prouns, one of which contains the best source. One may state with confidence 
P* that the parameter value (fraction effective) of each eliminated source is less 
than that of the best rorulation. Thus, when comparing 5 vendors with 10 sample items 
from each vendor, the best vendor has a 90% chance of being among those vendors having 
up to more failures than the vendor having the fewest failures. For example, if a 
vendor has 9 samples »assing, include all vendors having 9-) or 5 samoles passing in 
the better grour. If 311 vendors have 5 or more samoles passing, they are all in the 
better greur. Necte: One to all vendors may be included in the "best" group depending 
directly on the sammling results and indirectly on the true difference between them. 
Thus, this is a coarse method of screening. This procedure may be followed by the use 
of Procedure I if further screening is necessary. In doing this, it is assumed that 
the experimenter will use new observations; otherwise one has to be concerned with the 
effect on the rrobability of a correct selection of the procedure of carrying out two 
tests on the same data. 


TABLE I 


bability P* of a correct selection when the difference d between 

least d*. The four values in each cell correspond to the probability 

levels F* = 290, 295, -99,. respectively. 


a* fractional difference you wish to detect 
2 3 3 
542 135 60 3h 5 9 5 
1082 120 67 29 16 10 
3 i) 31 17 8 5 3 
8 5 
1 327 1k5 81 35 20 12 

59 ho 10 4 
BE 
¥ we 3 
5 10 | 606 152 g 38 17 10 6 
1803 Ns 198 111 is 26 16 


To annly Table II you must: 


1. Decide on the orchability level P* for the minimum vrobability of a correct 
selection. 

2. Record the maximum number of sammles "M" passing the test from any one cf the 
several sources or vendors being compared. 

3. Find the value of "d" in Table II for the samrle size "n", the number of 
vendors "k" and the nrobability level Px, 

4. Subtract the value of "d" fran the maximum number of samples that nassed 
from any one source, 

5. Group all the sources or vendors together that have ("™" - "d") or more 
samples passing. These are the best sources and P* is the probability that 
the very best source is in the "better" group. 


i 
d 
& 
} 
4 
ie 
ge 


430 1961 ASQC CONVENTION TRANSACTIONS 


x Number of Populations or Sources Being Compared 

2 5 © 56 © 
10 4 5 5 5 5 5 6 6 
H 5 Ss 6 6 6 6 6 6 
4 5 6 6 6 7 7 
20 4 6 8 8 8 8 
6 8 8 8 = 9 9 
8 9 9 10 10 10 
i” 5 8 9 9 9 3 10 10 
6 9 10 210 10 nu 1 n 
y 35 5 8 + 9 10 10 10 n nu 
6 68 9 10 10 n n n 12 
7 1. uu n 12 12 12 13 13 
45 6 9 10 n n nu 12 12 12 
8 2 n 12 12 13 13 13 YY 

6 n n 12 12 12 
8 12 13 13 13 4 


2 
Procedure IIIB 


This orocedure applies when the past history from one vendor or source called 
"control" is not known and we wish to find other sources that are "as good as" the 
control. This procedure guarantees with probability P* that all population or sources 
as good or better than the control are classed "as good as" the control. Thus, when 
comparing 5 vendors (one of which is the control) with 10 sample items from each ven- 
dor we have a 90% chance of correctly classing vendors "as good as" the control if we 
class vendors having | or less failures more than the control "as good as" the control. 


Thus, if the control hes 9 items (X,) passing the test, all sources having 9-l 
or 5 passing are classed "as good as" the cortrol. Note: (ne to 211 spources may be 
classed "as good as" the control depending on the sampling results obteined and in- 
directly on the true differences existing. This is a coarse method of screening. To 
determine which among the "as good as" sources is really best, use the sample sizes 
required in Table I for the final screening based on new observations. 


To apply the procedure, Table II is used: 


1. Decide on s probability level P* for the minimum probability of a correct 
selection. 

2. Record the number of items (X,) passing the test in the control. 

3. Find the value "d" in Table II for the sample size "n", the total number of 
sources "k" and the probability level P#, 

4. Subtract the value of "d" from the number of items "X," that passed in the 
control sample. 


TABIZ II 
Values of "a" for Procedures II and IIIB to be used where there is a 
common number "n" of observations from each of "k" populations. The 
two values in each cell correspond to the probability levels P* = .90 
and 2955 respectively. 
j 
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5. Group all the sources or venders as "as good as" the control that have "X,-d" 
or more items passing the test. 


2 
Procedure IIIA 


A procedure to compare (with probability of a correct selection P*) several (k-1) 
sources or vendors with a source (control) having a known fraction effective (P,) when 
(n) items are tested from each of the "k-1" sources. 


Use this when the part history fran one vendor or source called the "control" is 
known from incoming inspection records, factory rejects or field failures and we wish 
to find other sources that are "as good as" the control. Thus, if the fraction effect- 
ive of the control is .90 (percent defective 10%) and we are comparing it with two 
other sources (k = 3) with 50 sample items from each of the other two sources, and if 
we desire a test with probability P* = .90 of s correct selection then 1 or more items 
must pass the test for either source to be classed "as good as" the control. The table 
for Procedure IIIA has been computed only for k = 2, 3 and 5. Note: In all cases, the 
value of k is the total number of populations involved and includes the control. 


in Table IIIA an entry of zero indicates that the number of observatims taken was 
not enough to separate out any inferior populations on the prescribed probability level. 


To apply Table IIIA you must: 
1. Record the number of items from each of the k-1 surces passing the test. 


2. If the number of items passing is as large as or larger than the decision 
value in Table IIIA then class the source as "as good as" the control. 


2, 3 and 5, respectively. Pt = .90 


Each 
& 
COM COO 000 
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Smallest integer value "M" required for Procedure ITIA for selected values ae 
of "n*, Po, "k* and P¥. The three values in each cell correspond to “k* = ba, 
. 
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TABLE IIIA 
Smallest integer value "M" required for Procedure IIIA for selected values of "n", Po, 


"k" and P*. The three values in each cell corresrond to "k" = 2, 3 and 5, respectively. 
Pee 295 


0 0 1 1 2 3 
0 0 1 1 2 3 
j 10 ° ° 1 2 2 3 H 6 7 
c 0 ° 1 2 3 5 1 
° 1 2 3 5 ~ 
° 1 2 5 7 9 h 
° 0 2 3 5 8 10 
2 | o 1 3 4 6 33 16 
0 1 2 10 2 15 
3 » 3 5 9 4 
0 2 8 n 17 20 
° 1 3 3 8 10 13 16 19 
1 3 7 9 15 19 
50 2 10 2h 30 3 
1 13 iB 
1 12 17 33 ho 
3 


SAMPLE SIZES FOR OBTAINING REPRESENTATIVE SAMPLES 
Measurement Dats: 


A representative sample is a sample which faithfully reproduces the frequency 
distribution of the parent population. The degree of representativeness is a measure 
of how faithfully the sample reproduces the parent distribution. For practical con- 
siderations one has to limit the number of points at which the sample is compared with 
the parent distribution and also put 2 reasonable limitation on the degree of repre- 
sentativeness demanded. 


The tables described here divide the population into two cells and determine 
representativeness relative to these two cells. The two cells may be formed by divid- 
in g the population in half by the median or by taking two percentiles symmetrical 
about the median such as the 10th and 90th, or 20th and 80th and paying attention only 
to the two tails of the distribution formed by these percentiles. 


The faithfulness of the reproduction is given by B*, which is chosen so that with 
preassigned probability P*, a fraction between p-B* of the p+B* of the sample is below 
the population percentile p and also another fraction between p-B* and p+B* of the 
sample is above the population percentile l-p. 

The sample sizes are completely independent of the shape of the parent population. 


Choosing a Repres¢ itative Sample: 


1. Decide whether the sample is to be representative relative to the median or 
the taile of the distribution. 
a. If relative to the tails decide how far out in the tails, 10th and 90th 
or 20th and 80th percentiles. 
2. Decide how faithfully (B*) the sample must represent the population. 
3. Choose the prohability level P* at which this representation should take place. 


| 
| 
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Examples: 


A) In testing the effects of radiation on transistor reliability, it is thought 
that the tails of the distribution are of most interest. Should you desire 
to be 99% sure that 5% to 15% of the sample be below the populstim's 10th 
percentile, the required sample size is 220 transistors. P¥ = .99; BY = .05; 
10. 


B) In testing the temperature coefficient of resistors, it is desired to have the 
sample centered about the population median so that you are 90% sure that LS¢ 
to 55% of the sample is above (and also below) the median. In this case, a 
sample size of 251 resistors is required to provide the stated assurance. 
Pe = .90; p= .50. 


TABIZE V 

Minimum Number of Units Required for a Sample to be Representative 
50th Percentile 20th or 80th Percentile 
= 0.50) (p_= 0.20 or 0.80) 210 or 0.90 
| 10 ].15].20]] | .05] | .05 | .10 
1,051] 31) 5] S| 2 662} 12} 7] 6] 11} 355] wh} 2 
1,700] 60) S| S|] 1,062] 32] 7] 6) 11] Soo! 2 
2,600] 100} 20] 8] 1,662] 52] 10] 9] 900] 20] 12 
3,251] 120] 25 | 1] 2,062] 72] 10] 9] ho} 12 
4,051; 151] 35 | 91) 2,562) 92] 20] 12] ho} 2 
5,100) 191} | 17] 10 3,262) 120] 27 | 12] 60; 2 
90 6,700] 251} 60 | 26 | 13 || b,262] 160] 37 | 15] Sii2,355| Bo} 2 
095 |} 95551] 371 | 90 | 37 | 20 6,100] 50 | 20] 3,400] 120 | 10 
+99 16,500 | 651 | 160 | 71 | 39 10,562} 420 | 100 | LO} 5,900] 220 | 15 


: 


SELECTING POPULATION WITH REST mepran4 


We wish to compare two suppliers of the same type of transistor for I, (reverse 
saturation current - collector to base) and need to know the number of sample units 
to be tested for I,,, from each sunplier (population or parent distribution in statis- 
tical language) in order to determine which supplier is best. 


let us assume we wish to be 95% sure of choosing the better source and our 
definition of "better" is that population whose cumulative distribution is at least 
ad better at all voints within an interval + d* about the median. In this case we 
may desire that the better population be 15% better (which for I,,.) means lower) than 
the poorer povulation in an interval which extends from the 35th percentile to the 65th 
percentile cf the better population. 


| 
| 
| 
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The table tells us a sample of 63 units is required fran each source. 


To find the median, the measurements of I,,. from each source should be ranked in 
numerical order. The median measurement is the 32nd in numerical order - an equal 
number of measurements are above and below it. Note that Table IV, on page 10 has 
only odd sample sizes. 


To 


1. 


2. 


3. 


apply this table, certain decisions must be made: 


Select the probability P* of correctly selecting the population with the 
"better" median. Better may mean higher or lower depending on the design needs. 
The cost of the test increases repli with increasing P* so do not ask for 
95% or 99% accuracy unless you are willing to pay for it. 

Choose the mimimum distance d* between the percentiles that it is economically 
worthwhile to detect. This distance is in terms of the difference between 

the cumulative distribution functions through the chosen interval. As the 
sample size is an inverse function of the distance d*, it is best to choose 
d* as large as vossible. 

At the intersection of the distance d* and the probability p* in Table IV, 

we find the sample size which must be tested fram each population. 

The population producing the sample with the higher (or lower) median is the 
better population. 


TABLE IV 
pe 205 | -15 | .20 
50 21; 5] 3) © 
055 9) 3) 2 
60 47} 3 
065 67] 7) 5 
+70 
75 129 | 33] 15) 7 
60 181) 45} 19) 
85 253 | 63 | 27) 15 
290 363 | 91 | 39) 22 
095 567 | 63} 35 
99 1089 |271 |127| 69 
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HUNTING AND TAMING THE WILD OBSERVATION 


R. B. Murphy 
Bell Telephone Laboratories, Incorporated 


Introduction 
A. Definitions 
1. Extremes 
2. Mavericks, sports, and rogues 
3. Outliers 
B. History 
1. 19th century developments 
2. Later developments 
Practical Questions 
A. Direction of Attack 
1. How far to look 
2. Prevention or cure 
B. Risks 


1. Formal statistical errors 


a. Errors of the first and second kinds 


b. Errors of a third kind 
2. Hidden errors 


a. Wrong significance levels 
b. Masking 


c. Protective coloration in factorial designs 


3. Consequences of error 

C. Action 
1. Excluding, replacing, rescoring. 
2. The search for cause. 


a. Definite knowledge 
b. High likelihood 


3. Further exploring 
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III. Test Technique 
A. The Search for Normality 
1. Lack of robustness in statistical tests 
2. Transformation of data 
B. Specific Techniques 


1. Dixon* 
paradigm: (x5 - x, )/(x, - x;) where 


2. Grubbs** 
paradigms: 
n (x, -xF 
for 1 outlier — where 
s (n-1) 8 


= and = 2(x,-%); 


(for 2 outliers) 


28° (n«2) 


where X, = (x, + x,)/2 


C. Other Developments 
1. <A new approach: Anscombe*## 


2. Neutralizing protective coloration: Daniel*### 


*See Dixon and Massey, Introduction to Statistical 
Analysis, McGraw-Hill, ond Edition, pp. 75-078. 
*#F, E, Grubbs, "Sample Criteria for Testing Outlying 
Observations", Annals of Mathematical Statistics, 
Vol. 21, No. 1 (March, 1950), pp. 27-59 


*##F, J. Anscombe, "Rejection of Outliers", Technometrics, 
Vol. 2, No. 2 (May, 1960), pp. 123-147. 
*####C, Daniel, "Locating Outliers in Factorial Experiments", 
Technometrics, Vol. 2, No. 2, pp. 149156. 
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DESIGNING VALID EXPERIMENTS 


Charles A. Bicking 
The Carborundum Company 
Niagara Falls, New York 


INTRODUCTION 


The experimental approach to our environment is a well-ingrained 
human habit. We might observe that it is a lazy man's way of avoiding 
the necessity of thinking things through. However warranted the observa- 
tion may be with respect to many of our day to day activities, experiment 
ation is a strong and necessary support to theorizing in technology, where 
principles are grasped only darkly through our unaided senses. We might 
express concern that experimentation is so frequently only loosely con- 
nected with theory. We might not get very far on that tack either, be- 
cause®, as survivors of Nature's great experiment, most people seem to have 
the attitude that all the inept experimenters were eliminated when our 
would-be ancestors were testing the edibility of poisonous berries. We 
detect a cavalier attitude toward experimentation whether the outcome is 
likely to be lethal or not. We no sooner begin to feel good about the 
progress in use of experimental design throughout industry than we receive 
a shock from a communication like the following one: 


"The time involved plus the cost of testing in relation 
to the value of the information obtained pretty weil 
precludes the setting up of test patterns based on 
statistical plans. Usually, we hold all but one factor 
constant because it is the variable that we are usually 
interested in. Consequently, we do not often get a 
exotic in our methods as might possibly be desired. 


Such an attitude is based on unbounded confidence in an inate ability to 
plan experiments. 


The paradox involved is that, like Nature, whose method of experiment- 
ation, natural selection, is careless of time or materials, the intuitive 
approach is so often unconcerned about the relationship between time and 
cost factors and the value of the information obtained. Neither by Nature 
nor by training have we been given to understand the real character of the 
physical world. It is only slowly that science and technology have awaken- 
ed to the fact that it is not an exact but a probable world. In the face 
of the complexity of modern technology, our culture (the fault lies deeper 
than our educational system) has produced attitudes which have been alter- 
ed little from those of the berry tasters. 


A VIEW OF THE SCIENTIFIC METHOD 


We hear a lot about the scientific method. As we understand its 
application to experimentation, it involves a number of steps which appear 
to be unpopular with many people. For instance, there is the insistence 
on formal, logical thinking about the pattern of experimentation. Certain 
intellectual groups have been trained in logic, but not the technologists. 
They are being forced into it by the machine, perhaps too lately. We 
already have bionic machines, with internal logic, which are self-adaptive, 
making adjustments in non-directed (somewhat random) manner by a process 


called "free learning 


Then, there is the insistence on an understanding of the theoretical 
bases and the assumptions underlying the experiment. This involves recog- 
nition of experiment planning as being not a substitute for thinking, but 
an adjunct to thinking. It is a very lame excuse for poor experiment de- 
sign to claim that time couldn't be spared for a proper one because the 
problem was one for which the boss expected the answers the day before 
yesterday. 
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Not infrequently, we come upon a person with the attitude that it is 
better to begin anew than to base experimentation on what has been done 
in the past. There may be some excuse for this because we have only begun 
to concern ourselves about efficient information retrieval. However, a 
scientific approach to experiment design implies taking full advantage of 
background data to help decide how much work to do or when to stop.e 


Finally, the scientif‘c approach requires analytical skill, especially 
a gift for selection of fruitful questions to ask of Nature. For example, 
our berry-eating ancestors may have experimented with the relative digest- 
ibility of white or black pebbles with less drastic immediate effects than 
eating poisonous berries but with very little satisfaction as far as 
nourishment was concerned, 


The scientific method in experimentation points not to a few pat 
patterns of testing but to general principles which apply to all aspects 


of the work, (2 
HOW EXPERIMENTATION WAS IMPROVED IN THE PAST 


There are several ways in which experienced experimenters have attempt- 
ed to improve the quality of experiment design. One way was to attempt to 
control all the variables not of interest to the experimenter. This ap- 
proach broke down when the control was inadequate or when it could not be 
achieved at all. It also failed completely if unexpected or accidental 
variation arose. 


Awareness of the shortcomings of the above approach has led to depend- 
ence on balancing and randomization to achieve the desired results. 


Related to the concept of control of extraneous variables was the 
concept of changing the variables of interest to the experimenter one at 
a time while all the others were held constant. Many otherwise very 
capable experimenters seem unaware of the limitations of this approach 
although the sharper contrasts and the new information on inter-relation- 
ships among variables to be obtained simply by varying the experimental 
pattern scarcely need demonstration. 


Another approach to improving experimentation was one which is very 
sound and which should be followed at all times. This was to analyze the 
sampling and testing techniques and to improve their precision prior to 
use. In every experiment, sampling and testing refinements should match 
the requirements of the situation. 


Finally, another way used to improve an experiment was to increase 
the size of the experiment, that is make a larger number of repeated de- 
terminations under each set of conditions. Presumably, if we could afford 
to experiment without limit, eventually we could accumulate enough data so 
that the grand averages which we then could compare would disclose even 
minute differences in the effects of the factors we had varied. 


EFFICIENCY IN EXPERIMENTATION 


In a strict sense, the most efficient design is one in which the error 
variance has been minimized. In other words, an efficient design is one 
in which the variation in factors other than those under study has been 
controlied at a very low level. It is common to compare the error variance 
of a particular design with the variance that would have been expected had 


another et) been usea,(3) Por example, Cochran and Cox provide c ara-~ 
tive tables and Fisher discusses methods of estimating efficiency. 


In a broader sense, efficiency may be taken to mean the degree to 
which the information derived from an experiment has been maximized for a 
given amount of work. From this angle, efficiency embraces several factors 
in addition to the relative size of the error variance. The number of 
experimental runs may be minimized; the comparisons that we wish to make 
may be sharpened; information on interactions may be enhanced; or the range 
of the factors studied may be increased. Measures of efficiency may in- 
clude speed, economy, and administrative considerations. 
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Perhaps one of the most important attributes of the newer experiment- 
al designs being applied in industry is that they are designs with a pur 
poses 


DESIGNS WITH A PURPOSE 


There are two extremes of attitude that must be overcome. One is the 
attitude that nothing really amounting to a design is required or that a 
stereotype design, learned from a text or a “cook-book", is good enough 
for all occasions. . 


On the other hand, there are those who fasten upon one technique, 
often a relatively complex one, and seek to apply it universally, even to 
their simpler problems. There have been many specialized techniques made 
available to industrial statisticians. The authors of most of these have 
made the limitations clear and most users have applied them sensibly. In 
the long list of pet approaches, however, there are undoubtedly some which 
should be accepted guardedly and none which should be applied blindly 
under all circumstances. 


If there is any danger, it is that the skill required to select the 
proper technique and to apply it correctly may be under-emphasized. Let 
there not be too much beating of the drums (the authors themselves are 
not always at fault) for new inventions. There is such a tendency to ride 
a hobby that those who have followed the development of industrial statis- 
tics may be able to associate names with the following mixture of excell- 
ent and indifferent techniques: multiple correlation, Latin squares (we 
have heard one well-known industrial statistician called a “pig-Latin 
square man"), partial factorials, random balance, loteplot, span-plan, 
method of steepest ascent, response surface method, cumulative sum chart, 
evop (or even "revop"), and many others. They all have their place. 


However, we recognize several separate and distinct purposes of ex- 
perimentation, each of which may call for a different class of design. 
Among these purposes are: 


(1) Making simple comparisons 
(2) Testing the significance of changes 


(3) Estimating the effects of changes, both in level of 
performance and in variation 


(4) Moving toward optimum conditions 


(5) Describing the relationships among variables by equations 
or plots 


This list covers the span from limited objectives to all-out objectives. 

A design is evaluated, not by its complexity but by its ability to solve 

a particular problem. Refined statistical designs are by no means the 

only ones that pay off. It sometimes happens that intuitive designs, not 
at all sophisticated statistically, pay off handsomely. In other instances 
the statisticel planning required is rudimentary because of the simplicity 
of the experimental situation. 


A design of great complexity standing alone may not be as desirable 
as a series of less ambitious designs that sequentially add to our know. 
ledge. It is common practice to use brief, preliminary experiments in 
scouting for primary effects, followed by more complete designs covering 
the important variables. A good design is one which lends itself to 
extension. 


Several examples follow in keeping with the thesis that every experi- 
ment must be designed with a- purpose, 


EXAMPLES OF DESIGN WITH A PURPOSE 


I. The vibrational problems in complex equipment, such as rockets, 
are well known to the developers and users of the equipment. It may be 
surprising to hear that vibrational problems plague manufacturers of as 
staple a material as paper. Breaking of the paper web in a high speed 
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printing press is annoying 12nd can lead to publication delays and even 
to breakage of the press. n an instance in which web breaks were 
occurring at an alarming rate, a systematic analysis was undertaken in 
order to determine the causes, One design for data collection involved 
ee ages | the web breakage rate on rolls of paper shipped by rail versus 
rolls shipped by water. A higher web breakage rate was found for rolls 
shipped by rail. The explanation offered was pre-stressing by the con- 
stant vibration during rail-shipment of the rolls of paper which had 
been wound under tension. The end of each roll was pasted down, main- 
taining the tension, The pre-stressing weakened the web. The solution 
was to wind the rolls under less tension and to allow plenty of slack 
in pasting down the ends. The web breakage rate on rolls shipped by 
rail decreased. 


sneri le II. Aero-space materials improvement is of interest to both the 
ary and industry. One of the ways to improve materials is to im 
prove specifications, There is a particular interest in adequate speci~ 
fications for some cf the newer non-metallic, ceramic materials, inclu» 
ding many of the carbides, nitrides and borides. It is helpful to obey 
certain principles in establishing specifications: simplify - both 
the design of a part and the choice of characteristics far measurements 
apply direct rather than indirect tests; avoid lengthy and costly fine 
ishing operations; and don't overespecify. In the nonemetallics, fin- 
ishing problems control, whereas in metals, dimensional stability cone 
trols; in the former, compressive strength controls, in the latter, 
tensile strength controls, The user will probably require evidence of 
process and of laboratory control, The latter, in particular, will 
require the design of experiments, These will be designed with the 
urpose of making a precise statement or certification of the nature, 
ith 9 confidence, it is certified that not less than 99% of the 
material delivered has a density (or some other property) of X g/cc." 
When valid experiment designs are not their source, the data available 
are usually inadequate. 


ast III, Power spectrum analysis is valuable for study of rocket 

ight data and equally valuable for analysis of testing of commercial 
materials ultra-sonically. A specialized application is study of the 
surface characteristics of coated abrasives. Purposeful designs are 
necessary to obtain data that can be subjected to available techniques 
of analysis, In this instance, te egy a method of measurement and a 
method for collecting the data tax the ingenuity of a combined team of 
engineers and statisticians. Optical projection techniques have been 
combined with manual tracing of profiles and counting of included areas 
or combined with electronic scanning with a digital read-out. Horizon- 
tal scanning and vertical scanning have both been applied. Photographs 
of profile and plan of the material have been substituted for optical 
projection. Contact photography, optical displacement of a 
negatives, densitometer readings of impressions on thermo plastic tape, 
reading of mirror-images picked up on film and x-ray photography are 
other experimental data gathering techniques that have been explored. 

i Other even more complex methods of granularity measurement have been 
proposed, The resultant jata is often in the form of waves of very 
low frequency. Keen experiment design is needed to collect the informe 
ation in useable form, 


att IV, Certain kinds of experiment designs lead to particular methods 

of data analysis. For example, a factorial design leads to use of 
analysis of variance or response surface experimentation leads to matrix 
solution of regression relationships. When factorial experiments are 
applied to some problems, the effects of the independent variables on 
variances rather than on means are of interest. In this instance some 
additional treatment of the data such as transformation to logarithms 
may be desirable prior to use of the analysis of variance. In response 
surface experimentation, the analytical techniques lend themselves to 
variable measurements but not, at least at present, to psycho-physical 
attribute data, For example, in an experiment to optimize the density 
of a new abrasive material, two responses were recorded: measured 
density and a visual classification of porosity on an arbitrary scale 
from one to four, Response contours could be computed for the measured 
density. However, it would have been improper and misleading to have 
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analyzed the porosity data in the same way. Design with a purpose puts 
emphasis on planning for collection of data in appropriate design pate 
terns and with suitable method of data analysis in mind, 


Example V. A typical experiment involving dimensional restrictions is 
the sampling of bulk material as in a box car. A grid pattern is 
super-imposed on the plan of the car and a sampling thief is inserted 
in the material in the car at each grid intersection, Duplicate sam- 
ples should be taken at each location to provide a measure of experi- 
mental error. Sangnaee and crosswise arrays of measurements corres- 
pond to columns and rows in a two factor analysis of variance. The 
purpose is to detect any heterogeneity in the material, due to loading 
practices, or segregation of particle sizes, due to vibration during 
transportation. In some instances, the possibility of differences at 
different depths is also of interest, and a third dimensional factor 
is added to the experiment. Since dimensional locations are fixed 
one cannot get away without duplicate samples. As an end result of 
such a procedure an economical sampling plan, which will give desired 
protection against off-grade material, may be selected. 


Example VI. A more difficult problem involving dimensional restrictions 
arises in the characterization of sheeted materials or in sampling of 
these Sages of materials for process control, Certain types of grind- 
ing wheels are pressed from material which is a fibrous sheet in which 
abrasive grits have been embedded during the manufacturing process. 

A typical problem may be determining weight per unit of material died 
out with specified dimensions, The material is supplied in rolls from 
which sheets are cut. Discs are then died out from the sheets, The 
weight variations that are of interest include variation across the 
sheet, in adjacent positions in the long direction, at the extreme ends 
of the cut sheets, from sheet to sheet within a roll, and from roll to 
roll. Obviously, only one sample can be cut from one position, so that 
replication in the true sense is never attained. Cause systems affect- 
ing weight across the sheet and in the length direction are different. 
The variation across may consist of a bias, rather than random varia- 
tion, although random shifts in weight may be characteristic of some 
making machines. Variations in the length may be random, or cyclic 
(from a few inches to many yards), or may indicate long trends (even 
seasonal ones). In planning experimental data, the natural tendency 

is to take samples from fixed positions. For some purposes this is 
desirable, Consideration should also be given to random sampling and 
what this will do to the model of the experiment. Proper selection of 
sampling interval is important. It may be desirable to take into ac- 
count principles applying to power spectrum analysis, for example, the 
'Nyquist' sampling interval which requires equidistant sampling at least 
twice in the interval of the shortest harmonic of interest. There is a 
'Bicking' interval which calls for random sampling at least once in 
every cycle or if the cycle frequency and shape are not known, at least 
as often as the expected number of cycles. The Nyquist sampling implies 
Fourier analysis, random sampling a least squares approach, or auto- 
correlation, None of these methods of analysis are generally associated 
with sampling on a factorial plan. 


Example VII. If we are going to use indirect measurement (for example, 

ensity of a molded silicon carbide body), rather than a direct meas- 
urement (per cent SiC in the body) we are asking for more work when we 
start to combine experimental results for the purpose of writing a 
specification or making a certification statement. So, it becomes of 
importance to be purposeful in the designing of experiments to collect 
data with the end-use in mind. The same thing is true of combining 
variance data from different sources. Combining variances, and particu- 
larly estimating degrees of freedom for the result, have involved labcr- 
ious computation in the past. The problem can be avoided, in the first 
instance, by using only direct measurement where this is at all possi- 
ble. Explicit designs to measure variation from several sources sim- 
ultaneously are of help in the second class of situations. Tables of 
Tolerance ~ Limit Factors for Normal Distributions now cover the situa- 
tion where the total variance is made up of components from various 
sources, With the increase in contractual requirements for the certi- 
fication of quality, it is important to design experiments taking factors 


ae 
i 
A. 


444 1961 ASQC CONVENTION TRANSACTIONS 


such as these into accounte 


Example VIII, In some instances, it is desirable to know all the relation- 
ships among two or three independent experimental factors as they affect 
one or more dependent factors or responses. For example, in developing 
a free-flowing, storable temporary binder for use in the molding of 
vitrified grinding wheels, it was desired to determine the influence 
of size of grit, of per cent bond, and of inorganic d@msity, each over 
the usual range of manufacture, These variables could affect several 
finished product properties, including hardness, density, and response 
to ultra-sonic vibration. A three factor central composite rotatable 
design was used with six replications of the center point (paired 
randomly throughout the vende It is possible to obtain complete sets 
of equations representing this system for each of the three responses. 
Also, a plotting routine on an electronic computer prints out for each 
product property, the response contours for each of the three pairs of 
independent variables. The whole bonding system is much better under~ 
stood than it ever was before. Optimum formulations may be picked from 
the charts and combinations resulting in poor product quality can be 
avoided, 


CONCLUSION 


A valid experiment design is one that applies statistical principles 
for the achievement of a specific purpose. There are several very good 
texts availabie as sources of designs, (3) (4)(6) Skill must be developed 
in selection of the technique and in its correct application. The design 
may be very complex or it may be so simple that no special skill was re- 
quired to plan it. Nevertheless, the purpose of the design will be served 
best if it matches the situation and if it lends itself to statistical 
analysis. 
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RESPONSE SURFACE TECHNIQUES AS A STATISTICAL APPROACH 
TO RESEARCH AND DEVELOPMENT IN ULTRASONIC WELDING 


Elizabeth G. Ross, Statistician 
Aeroprojects Incorporated, West Chester, Pa. 


INTRODUCTION 


Statistical designs and analyses have been very useful at Aeroprojects in 
research on and development of a new and unique process, ultrasonic welding, 
Particularly useful are the methods of fitting response surfaces and the Peeoting 
of optimum operating conditions as developed by Box & Wilson, Hunter and others, 
These methods are applied to experiments in welding programs, and involve the use 
of specific statistical designs such as factorial or fractional factorial, hexag- 
onal, central composite, etc., i.e., designs of two or more variables, The statis- 
tical analysis of the results provides an estimate of the response surface or con- 
tours, and shows the relationships between the independent results (strength of 
weld) and the controllable factors (power, clamping force, etc.). Such an analysis 
also gives the direction of "steepest ascent" of the contour that is used in esti- 
mating the best levels of the variables so that, in subsequent experiments, still 
greater strength of welds may be obtained, Some specific examples of these methods 
will be presented together with the surfaces fitted to the data, 


All experimentation at Aeroprojects is accompanied by standard quality control 
techniques, We recognize that the experimenter"often neglects the inherent causes 
of unreliability. He is inclined to believe that reliability is an ability -- where- 
as, in fact, it is only a probability"(11)*. By means of statistical methods, the 
intuitive type of reasoning which an intelligent experimenter might apply in drawing 
inferences from data is made objective and precise, 


ULTRASONIC WELDING 


Ultrasonic welding is a non-fusion, solid state process for joining similar and 
dissimilar materials by the introduction of high-frequency vibratory energy into the 
area to be joined (13), A strong metallurgical bond is produced without the aid of 
solders, fluxes, or filler metals, without the external deformation that character- 
izes pressure welding, and without the cast metal zones associated with resistance 
welding. The process has been successfully applied to spot-type welding, spot-seam 
welding, continuous-seam welding, and area welding (15, 16, and 17), 


Several types of machines are available. Figure 1 illustrates a precision 100- 
watt capacity ultrasonic welding machine used for joining foils, fine wires, and 


semiconductors (such as silicon), This unit is particularly applicable to the delicate 
welding problems encountered in the transistor construction field, Figure 2 shows the 


most recent and largest ultrasonic welding machine designed for commercial use.** 
This welder is capable of welding structural gages of such metals as aluminum and 
stainless steels. 


RESPONSE SURFACE TECHNIQUE 


A brief explanation of the response surface fitting techniques may be helpful. 
With two controllable independent variables (such as power and clamping force), and 
a dependent resultant variable (such as shear strength), it may be possible to obtain 
several types of configurations or surfaces such as shown in Fig. 3, where shear 
strength is represented by the contour lines graphing the surface, The response 
function of two variables represents a sketch of a three-dimensional function in two 
dimensions, This simplified presentation of a contour map, which projects sections 


“Numbers in parentheses refer to the list of references at the end of this paper. 


“these ultrasonic welding machines are manufactured and sold by Sonobond Corporation, 
a Subsidiary of Aeroprojects Incorporated, Further information on these and other 
ultrasonic welding machines can be obtained from Sonobond Corporation, 202 W. Market 
Street, West Chester, Pennsylvania, 
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Fig. 1: SONOWELD® MODEL W-100-TSL Fig. 2: SONOWELD® mopEL W-h000-FSR 


of the three-dimensional surface on to a two-dimensional plane, appears easier to draw 
and simpler to understand, 


The Figures are typical response surfaces on ultrasonically welded specimens that 
have been estimated from experimental results obtained by using statistically designed 
experiments such as those presented in Fig. ). 


The design in Fig. LA is a 2 x 2 or 22 factorial, with two levels of each of two 
variables, Figure 4B is a simple cube or "Box" using two levels of each of three vari- 
ables, sometimes called a 23 factorial. These are first-order experimental designs 
with replicated center points. 


When it is necessary to determine the response as a function of several controlled 
variables over « wide range of these variables, a second-order experimental design 
should be used, Two such designs are represented by Fig. C and Fig. UD. 


For a more detailed explanation of the technique of response surface fitting, 
including the Doolittle Technique (4), see Davies, "The Design and Analysis of Indus- 
trial Experiments" (2). 


ADVANTAGES OF USING A STATISTICAL DESIGN EXPERIMENT 


According to Besse Day (6, 7, and 8), the chief characteristics of statistical de- 
signs are the inclusion of mltiple factors within some test at more than one level, 
and the randomization of all possible sources of variation, including order of testing. 


The resultant advantages that these characteristics provide are as follows: 


(a) A valid base for computing the numerical uncertainty of inferences drawn from 
the experimental data 

(b) A gain in the precision of estimates 

(c) A greatly enhanced scope of conclusion 

(d) A substantial saving in cost 

(e) An increased sensitiveness leading to the detection of small real differences 
which otherwise might not be evident, 


It is also possible to measure not only the effects of factors at various levels, 
but also the joint effects of the factors. If we use the classical approach of inves- 
tigating one factor at a time, we may be measuring effects and assigning their cause 
to the specific variable under investigation when in reality we are measuring the 
influence of variables which are not as yet known. Dr. Hunter (l) has given a graphic 
illustration of the favorite and classical approach of varying one factor at a time 
while holding all others constant, He also has shown that in using this approach, the 
experimenter may completely miss the maximum if a certain type of response surface 
exists. For instance, if the surface shown in Fig, 3B exists, the experimenter may 
use a constant, 1,0 watts and vary the clamping force until a maximum of 60-lb shear 
strength is obtained at about 200-lb clamping force, Then, keeping the clamping force 
constant (at 200 1b) he my increase power by steps from 10 watts to 600 watts, thus 
reaching a maximum shear strength of 87 pounds, However, a 100-lb shear strength level 
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Fig. 3: TYPICAL RESPONSE SURFACES OBTAINED 
WITH ULTRASONIC WELDING DATA 


could be arrived at by use of a statistical design with "steepest ascent" approach, 
Thus, by means of planning, and by establishing a statistical design, it is possible 
to gain increased precision, to broaden the scope of the conclusions, and to obtain 
more information from the same amount of experimentation (or a decreased amount of 
experimentation with resultant saving of time and money), 


RESPONSE SURFACE TECHNIQUES AS USED IN ULTRASONIC WELDING 


An Ultrasonic Welding Program at Aeroprojects, whether research, application engi- 
neering, pilot-plant services, or production of welding equipment, involves the fol- 
lowing procedures: 
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1. Statistical design of experiments 
2. Statistical analysis of data 

3. Routine quality control. 

1. Statistical Design of Experiments 
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In general, the experimenter and the statistician must precisely state the objec- 


tives and possible outcomes of the experiment, 
have a knowledge of the details of the proposed project. 


It is important that the statistician 
Then the experimenter and 


the statistician, working together, screen all possible variables and eliminate all 


but the most important. 


The cardinal rule, as stated by Crawford (10), is to use sta- 


tistics in the panne stage. The experimental design, i.e., the number, spacing, 
nterre 4 


replication, 
correctly chosen, 


lationship of the individual experimental trials, 


In choosing the pattern for each experiment, after the scope and objectives have 
been formlated, the possible experimental testing levels for major variables are estab- 
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Fig. h: EXAMPLES OF STATISTICAL EXPERIMENTAL DESIGNS 
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lished jointly by the experimenter and the statistician, In judging the amount by 
which a factor should be varied in the experimental design, the experimenter is at- 
tempting to get as near as possible to the maximm response indicated in the relation- 
ship between the variables. Of course, when a second experiment is carried out, the 
statistician has some basis for determing the relationship of the factors, With ultra- 
sonic welding, the relative scales for each variable are fixed by the judgment of the 
experimenter and statistician when they decide the relative amount by which the factors 
will be varied in the experimental design. Skillful choice of the amount by which the 
factor should be varied will considerably reduce the amount of subsequent work, It 
mst be emphasized that the choice of ranges to be used on each variable is by no means 
governed by rule of thumb; rather, experience and judgment must play an important part 
in this procedure, The reproducibility of the variables should influence the choice 
of scales; e.g., the better the reproducibility, the smaller the intervals, In ultre- 
sonic welding, the properties of the material being welded are a deciding factor, 


2. Statistical Analysis of Data 


Statistical Analysis of the data includes the fitting of response surfaces, test- 
ing for "goodness of fit", and finding the proper levels of each of the variables used 
that result in the greatest strengths possible with a good weld (this last uses the 
method of steepest ascent on the fitted contour to point to levels for further experi- 
mentation), This technique is best described by a few examples which show the designs 
used, the statistical process used in fitting the response surface by the Doolittle 
Technique, the resulting surfaces or contours in both eraphical and equation 
form, the testing of these surfaces for “goodness of fit", and the calculations that 
determine the direction of "steepest ascent", 


The path of "steepest ascent" technique assumes that a plane can roughly approxi- 
mate the true curved surface in a limited area, The tilt of the fitted plane can then 
suggest ~ direction of further experimentation. For the sake of brevity, a very 
simple 27 (3 factors at 2 levels each) experiment has been chosen to demonstrate the 
type of calculation used, The example gives a diagram of an experimental design where 
the vertices of the cube indicate the machine settings of power, force, and time, The 
four solid dots in the vertices of the cube show the half-factorial; the circles des- 
ignate the other half-factorial which could be used, The calculated example presents 
the actual machine settings for each experimental point as well as the coded machine 
settings, or X values, Also presented are the coded Y values (shear strength response 
values), the sum of the response measurements, and the differences for the two 
replicates, 


First-Order Experiment with Three Controlled Variables 


A 23 factorial design showing the division into two 
half-replicate designs. 


Mathematical model y = Ax,.* Ax, + Ax, + Ax, + 


Vector of Observations (Coded)* 


ear 
Factor Levels Matrix of Strength 
Xp x3 Independent Experimental 
Force, Power, Time, _ Variables Replicates 1b Error 
1b w pulse Noe 1 Noe 2 £y 
750 3200 1.5 2 & 91 91 182 
750 3200 10 1 79 80 159 1 
= 1.5 2 169 Error = 
7 200 1.0 1 1-1 103 23 
550 3200 1.0 : 80 80 160 86h, 
550 2h00 15 1-1 78 78 (156 o 432 
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Forward Doolittle: 
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Convenient procedure to solve simultaneous equations 


Design set up for easy solution of Coefficients (Code Y values) 


Ex,? Ex, x3 
Proceed by dividing all 
the elements by the lead- 
ing element, as follows: 


Row 1 
(Eq for b 


Row 2 
(Eq for b,) 
Row 3 
(Eq for b.) 


Row 
(Eq for b3) 


Using the coded parameter, the equation for the 


plane is as follows: 


0 
16 0 
1 0 
16 

1 


te) 
te) 
6 
1 


1230 = Zy or 16d, * Ob, + Ob, = 1230 
-4 = or 16b, * Ob, + Ob, = 


92° Expy or 16b, * Ob, = 92 
104 = or 16b, = 104 


1230.000 Sum of Squares 
76.875 = by by = (1230)(76.875) 
"Correction factor" 


-.000 
0.250 = b, b, = (-4)(-0.25) 


92.000 
5.750 =b, by = (92)(5.75) 


10.00 
6.50 = b, = (10h)(6.5) 
EStimate of the 


Coefficient 


Shear Strength Y = 76.875 - 0.25X (force) + 5.75X5 (power) + 6.5X3 (time) 
within the limitations of this first experiment. 


Regression Analysis Table: 


ss 
y 96,928.00 
by 94,556.25 
bos by 1,206.00 
Residual 1,165.75 
Lack of fit 733075 
Error 432.00 


Tabled Variance 5% 
Points of €°* Ratio 


16 

1 

3 402.00 

12 

183.4375 3.397 Pye 8 = 3.84 
8 54.00 


Observe that the lack of fit mean square is not significantly different 
from the experimental error mean square at the 95% probability level. 


The outcome 
a second experiment. 


of this experiment is merely to have some basis for designing 


Calculation of Path of Steepest Ascent and Subsequent Trials on the Path: 


*2, 
Seat Force Power Time 
(1) Base Level 650.00 2800 1.25 Average Y, my 
strength = 768.75 1b 
(2) Unit 100.00 400 0.25 (not coded) 
(3) Estimated Slope b (change -0.25 5.75 65 
in yield per unit) 
(4) Unit x b -25.0 2300 1.625 
(5) Change in level per 00 w “Lek 400 0.28 
change in Xo 
(6) The path of steepest 650.0 2800 1.25 
ascent represented by a 
series of possible trials 645.46 3200 1.53 
on it 641.2 3600 1.81 
636.8 4,000 2.09 
*Coded Resp (Y - Resp divided by 10) 
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The Doolittle solution of this design is a convenient method of solving the 
simltaneous equations that are used to obtain the response surface, Note that the 
design is set up with zeroes for an easy solution of the coefficients. The calcula- 
tions generally follow the procedure outlined in Davies (2). 


Experimentation with various materials and thicknesses has established that the 
relative influence of controlled variables and their appreciable interaction effects 
are specific for each type of weld (one relationship for Al to Al, another for Cu to 
Al, Ti to Ti, Ti to steel, etc.). A minimum of observation and a complete evaluation 
of weld quality by means of nondestructive x-ray, and measurements of weld area, cross- 
tension, percent deformation, and tensile-shear strength, points the way to subsequent 
trials from a first, fully designed experiment. The amount of repetition necessary 
is based on the known variability and on the cost of the first experimentation, 


Some specific experimental designs and resulting contours are included in Fig. 5, 
6, and 7, and illustrate the use of the methods of response surfaces in ultrasonic 
welding. 


Figure 5 uses a 3 x 5 factorial in order to obtain the contour indicated by the 
equation, The volume of experimentation and the calculations are quite detailed in a 
factorial with 15 experimental points. 


1700 
= 
i 1500 F- e 
1300 F- 


1 L iL 1 


1580 250 350 450 550 
Clamping Force, |b. 


363.8703+ 36.7666X, + II-7500X,- 32.8204X, — 5.4442X5 + 12.5500X, X, 
Estimate of Lack of Fit: Fo3,=lI7 (Fe30=1.50, 80%) 


Calculated Equation: 


Fig. 5: AN EXAMPLE OF A RESPONSE SURFACE OBTAINED FROM A FACTORIAL- 
DESIGNED EXPERIMENT SHOWING LEVELS OF TENSILF-SHEAR STRENGTH 
FOR 0,020-INCH COPPER (HALF-HARD) 


Figure 6 shows a rotatable hexagonal design (indicated by the dots), which gives 
the results at each machine variable in pounds tensile-shear strength. An experimental 
design possessing equal predictability in all directions from the center of the design 
at a constant distance from the center is called a "Rotatable" design. The center 
point is replicated more frequently than the peripheral points in order to provide an 
estimate of the experimental error. Replicating the center point also insures good 
predictability toward the center of the design, 


It is a great satisfaction to be able to determine whether or not the fitted model 
is a proper one, or how well the proposed mathematical model fits the data, The model 
presented in Fig, 6, is an excellent fit, F, = 0.1227, Tabled value of the F- dis- 
tribution (X = 0.50, = 1, and No * 10) baud?s 0.48973. Of course, this is a limit- 
ed experiment, and the F value from an analysis of variance compares the fitted sur- 
face variance, which has 1 degree of freedom, with the error variance which has 10 de- 
grees of freedom, 
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Fig. 6: AN EXAMPLE OF A RESPONSE SURFACE (ROTATABLE HEXOGONAL DESIGN) 
OBTAINED AS A RESULT OF THE BI-METALLIC WELDING OF 0,063-INCH 
COPPER TO 0,032-INCH ALUMINUM 


In the examples presented the machine settings indicated are not necessarily opti- 
mam settings; rather, the settings represent experimental problems that have been 
studied to determine those few controlled variables which have the most profound effect 
on ultrasonic welding, 


An interesting experiment with welding of molybdenum te itself (Fig. 7) evolved 
when a rotatable hexagonal design was used with five levels of clamping force and three 
levels of ultrasonic power, The contours of the first experiment indicated a signifi- 
cant fit of the second-order mathematical model, 


The region of wide variability and microscopic evidence of inferior quality weld- 
ments were concentrated in the peripheral experimental points of the hexagon, The 
first experiment indicated that crack-free welds of acceptable strength could be ob- 
tained in the center of the experiment, This indication was affirmed with a larger 
sample as well as the photomicrograph as shown in Fig. 7. A second experiment which 
involved a large sample at each point on an equilateral triangle within the original 
experiment verified the original data, The only region where high quality welds were 
obtained was within the elipse enclosing the center of the experimental region, 


Ralph Waldo Emerson once said: "Bad times have scientific value. These are the 
occasions a good learner would not miss." Poor quality welds in some regions provide 
as moch information as the actual knowledge of the region in which we can obtain good 
quality welds, 


Figure 8 illustrates how the experimental results can be presented to the experi- 
menter in a simplified and complete form, The response surfaces may be shown on a sepa- 
rate graph. This illustration is primarily for the experimenter, with the actual weld- 
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Fig. 7: A RESPONSE SURFACE (ROTATABLE HEXAGONAL DESIGN) OBTAINED 
AS A RESULT OF WELDING MOLYBDENUM TO MOLYBDENUM. OPTIMUM 
RESULTS WERE CONCENTRATED IN THE CENTER OF THE ELIPSE, 


ed specimens (A) mounted in relationship to the experimental design, The photomicro- 
graphs (C) provide complete information about the experimental region that results in 
poor quality welds, as well as the region that produces high quality welds, Part B of 
Fig. 8 presents the design levels and the various measurements obtained at each respec- 
tive experimental point, These various results aid the experimenter and the statisti- 
cian in establishing standards and in subsequent welding. 


3. Quality Control Techniques Used in Ultrasonic Welding 


Improvements in the ultrasonic welding equipment from time to time, in an effort 
to establish an acceptable standard, have been accompanied with continuous statistical 
quality-control charts on shear strength, Statistical tests of significance have shown 
that shear-strength measurements follow the normal distribution pattern, The selec- 
tion, control, and sensitivity of routine performance tests are considered and char- 
acterised by one or more parameters such as means and standard deviation, Low vari- 
ability in the tensile-shear strength measurements indicates that proper engineering 
has produced welding equipment which provides consistent, high quality welds that will 
function satisfactorily for the purpose intended, This low variability, as measured 
by the standard deviation, is induced by constant checking with control charts. In all 
‘instances, the points that fall outside the plotted control limit are investigated, In 
one case the component parts of a transducer-coupling system had to be re-worked be- 
cause the variation in welds it produced was greater than should be expected, This is 
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illustrated in Fig. 9, and 10, which show the results of data obtained on four identi- 
cal models of machines that were being tested to determine whether or not they would 
meet the specification limits established for this class of welders, Welder No, 2 did 
not meet this specification. The first testing resulted with a mean value of 790-lb 
shear strength, and a standard deviation of 177 1b. This unit was returned to the 
Fabrication Shop, and xfter modifications, the performance level of this unit, as in- 
dicated in Fig, 9, maintained a mean value (990 1b) equal to the best machine; also, 
this unit had the lowest standard deviation of the four machines tested, 
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Figs 9: MEAN TENSILE-SHEAR STRENGTH, 1b, FOR 
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Fig. 10: STANDARD DEVIATION OF TENSILE-SHEAR STRENGTH, 1b 
202),-T3 BARE ALUMINUM, 0,00 INCH 


SUMMARY 


Statistical techniques and designs in fitting response surfaces have been partic- 
ularly useful in research and development of ultrasonic welding at Aeroprojects In- 
corporated. These methods have been particularly useful in differentiating between 
the operating conditions that produce high quality welds and those which produce 
defective or cracked welds with wide variability in strength, 


* For aluminum spot-weld shear specimens of 0,00-in, thickness, the minimum averace 
strength required by MIL-W-6858A is 35 lb, I.. Fig. 9, the mean tensile-shear 
strength of 0,040-in, thick aluminum welded ultrasonically is indicated as 900 lb 
or over, which well exceeds the requirements of MIL-W-6858A, 
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QUALITY CONTROL IN 
RESEARCH AND DEVELOPMENT 


Robert G. Fitzgibbons, Manager, Quality Control Section 
Raytheon Company, Wayland, Mass. 


To ensure the quality of engineering model equipment quality control should be in- 
stituted during the research and development phases of a program. Minimum criteria for 
conformance to MIL-Q-9858, "Quality Control System Requirements" are presented as well 
as a suggested method of implementing the controls. 


INTRODUCTION 


During the past year, I have participated in the introduction of a forma] program 
of quality control in an electronics research and development (R&D) laboratory. During 
this period many questions regarding the appropriateness of a formal quality control 
program during R&D), and the proper means of implementing such a program have been con- 
sidered. The material which follows represents the writer's opinion on a somewhat con- 
troversial subject. It is hoped that the experiences presented will be of help to others 
similarly employed. 


HISTORICAL DEVELOPMENT OF QUALITY CONTROL IN R&D 


New products are conceived, designed, fabricated, and refined during R&D. Until 
approximately five years ago there was little "quality control" (in the sense that these 
words connote for Manufacturing operations) during the process of research and develop- 
ment of electronic products. Seldom were independent "quality control" agencies, apart 
from those doing product design, functioning to assure specification compliance during 
R&D. Many companies, with Quality Control Departments reporting at the same level as 
Engineering and Mamfacturing Departments, had some program for control of quality in 
R&D stages. Other companies had Reliability Departments reporting to Engineering with 
some responsibility for control of quality during R&D). However, with few exceptions, the 
intensity of quality control efforts were much less during the R&D program phases than 
during Manufacturing phases. Usually the practice was to have only enough quality con- 
trol during R&D to prepare Mamfacturing Quality Control personnel for the Manufacturing 
phase of a program. Intensive quality control efforts were largely confined to Manufac- 
turing. This situation has changed and is still changing in the electronics industry. 


In recent years the functional capabilities of electronic products have greatly in- 
creased. This increase has been accompanied by increases in equipment detail, limita- 
tions on size and weight, constraints on development money, and compressions in time 
schedules. Despite these factors product quality levels have had to be maintained and, 
in many instances, improved. 


In the past,it was possible to develop an experimental model, produce a set of draw- 
ings and specifications during the course of equipment development, prove out these draw- 
ings in R&D in the constructions of a prototype, and release a proven set of prints to 
Manufacturing. Manufacturing would then produce quantities of equipments for field use. 
The military customer would exercise control over the quality of this equipment by in- 
specting and testing after product completion, prior to shipment to the field. 


As products in Manufacturing became more complex, the military services realized 
they could not expect to exercise control by repeating contractor inspections and tests 
and began to specify minimum requirements for contractor quality control systems during 
the Manufacturing phases of their contracts. The most well-known such specification was 
the Air Force MIL-Q-5923, "Quality Control Requirements." 


As the rate-of-change of the state of the art increased, products became obsolete 
even during design. Expressed as a percentage, many more equipments in use in the field 
were produced as experimental units than was previously the case. As a result, the ap- 
propriateness of having quality control systems in effect during the course of construc- 
ting these R&D equipments in order to assure specification compliance was recognized. 
Not only this, but it was recognized that introducing quality control during the R&D 
phase of a program would considerably reduce the time from design to full production by 
contributing to the elimination of problems during the R&D phase that would have other- 
wise gone to Manufacturing. 
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As a result of the above recognition, within the last two years, the Department of 
Defense has issued a coordinated quality control specification, MIL-Q-9858, "Quality Con- 
oN ae Requirements," as a replacement to MIL-Q-5923 (USAF) and also MIL-G-14461 

ORD). 


Unlike MIL-Q-5923, this specification does not define one approach for Manufacturing 
and another for R&D. It would appear that it defines quality control system requirements 
for both R&D and Manufacturing, but, it makes no real distinction. However, there is 
more to having a specification accepted than its issuance and, at the present time, it 
does not seem to be uniformly accepted by industry and the military that MIL-Q-9858 per- 
tains to R&D. 


Also, since MIL-Q-9858 is considerably less detailed than MIL-Q-5923, there is mch 
judgment which can be exercised by both industry and the military in its —— 
The question can then be asked, "Does MII-Q-9858 pertain to R&D, and, if so, how should it 
be interpreted?” 


The conditions which apply for — control in R&D are somewhat different from 
those which apply in Manufacturing. New "minimum criteria" for quality control in R&D 
need to be created which permit the flexibility required in an R&D environment. 


There are many interpretations which must be made regarding the applicability of 
MIL-Q-9858 to R&D. As mentioned above, it is just not appropriate to apply the same 
quality control measures traditionally applied to Manufacturing to R&D. Nevertheless, 
this does not mean that with a little bit of common sense and the application of some 
Foe 4 applied exceptions, it is not possible to live up to the spirit of MII-Q 

in R&D. 


The activities which follow are those where the traditional mamfacturing approach 
to quality control, as typified by Phase B of MIL-Q-5923, may run head-on into the tradi- 
tional flexible engineering approach of R&D. Ways of preventing an impasse are suggested. 


Drawing and Change Control 


One of the most difficult problems to solve when one tries to apply Manufacturing 
quality control procedures to R&D is the problem of controlling drawing changes. MII-Q- 
9858, paragraph 3.3, Drawing and change control, states as follows: "A procedure shall be 
maintained to assure that the latest applicable drawing, technical requirement and con- 
tract change information will be available at the time and place of contractor inspection. 
Concurrently with the effectivity of revised drawings or changes, the contractor's change 
control shall assure that obsolete information is removed from all points of issue and 
use," 


During R&D in many companies, drawings are not usually under a change control system 
and do not come under a change control system until released to Manufacturing. When an 
engineer has placed a part in the Model Shop for construction or has placed a part with 
a vendor for construction, he sometimes retains the master or tracing of the part. If 
trouble arises in the Model Shop, he may mark up and initial the print in the Model Shop. 
Similarly, when a purchased part is being made to an R&D drawing, the vendor will often 
call the Engineer when problems develop and the Engineer may empower the vendor to change 
the print and initial it in his behalf. If the changes are not incorporated into the 
tracing before Inspection's print is made the parts will be checked to the wrong print. 
If the Engineer neglects completely to change his tracing, there will be a lack of corre- 
spondence between delivered product and documentation at the time of shipment of the engi- 
neering model and an incorrect print will be released to Manufacturing. 


These problems are, of course, not unique to R&D. They happen in Manufacturing as 
well. But, the fact that the prints are being used to construct parts or equipment for 
the first time results in a quantity of changes which is considerably above that in Man- 
ufacturing where prints are used which have already been upgraded through their use for 
the fabrication of Model Shop equipment. 


One answer to the problem is: "Well, just stop such practices from going on. For- 
bid the use of marked-up prints, require that all changes to prints in the Model Shop be 
incorporated through formal change control procedures, and require that all instructions 
to the vendor be routed through Purchasing and that the vendor be instructed not to change 
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anything until he has it in writing from Purchasing." There seems to be universal agree- 
ment among Engineers that such a process would cripple an R&D program. 


Some Engineering Departments think that permitting the Design Engineer to retain 
the tracings of his drawings during R&D and making him solely responsible for incorpo- 
rating changes on tracings may result in a smoother operation, a better correspondence 
between delivered equipment and prints, and a better set of prints delivered to Manufac- 
turing than if a system of change control were introduced. There is also feeling that 
he =A sa and time delays attendant with a formal program of change control would be pro 


What is the answer to this dilemma? While the full rigors of a Manufacturing 
control system are not appropriate to the R&D scene, nevertheless, some minimum change 
control system which puts some checks on the Design Engineer for change control is nec- 
essary. The following are the minimum criteria such a system should possess: 


1. Any change which is not made directly to the tracing must be "marked up" on two 
"change prints", and a revision number assigned to these marked-up prints. 

2. The effectivity date or model number of the change must be shown on the change 
prints. 

3. The change prints must be dated and signed by some pre-designated responsible 
person. 

4. One of these change prints must go to Quality Control. 


In this fashion, Quality Control will have the latest applicable information to 
perform its inspection, and can check later to see that the latest information has been 
incorporated into the tracing at the time of release. 


Receiving Inspection 


Another problem which arises when one tries to apply Manufacturing quality control 
measures to R&D) is how to control the quality of purchased material. MIL-Q-9858, para- 
graph 3.5.5 “Receiving Inspection" states, in part: "Subcontracted supplies shall be 
subjected to inspection after receipt, as necessary, to assure conformance to contract 
requirements. . « » « The contractor shall provide procedures for withholding from use 
all incoming supplies pending completion of required tests or receipt of necessary test 
reports, except that supplies may be released when under positive control. ... ." 


During R&D many companies do not require that purchased material be given a "re- 
ceiving inspection" as required during Manufacturing. The reason given for this is that 
the Design Engineer is buying the material for his model and should have the prerogative 
of deciding whether or not it requires receiving inspection. Another reason given is 
that the Design Engineer will inspect and test the material himself and therefore re- 
ceiving inspection constitutes a double inspection, the expense of which should be avoid- 
ed. Still another reason given is that the parts being purchased for R&D are new parts 
which represent an advancement in the state of the art and that Receiving Inspection 
doesn't have the know-how or the equipment to check such parts. This is closely related 
to the concern that Receiving Inspection personnel may damage delicate items purchased 
for engineering models. 


I believe that the Design Engineer should not have the prerogative of deciding what 
should or should not be inspected in Receiving Inspection when the item being purchased 
is intended for incorporation into a shippable model. The reasons for this opinion are 
that if purchased material is not examined in a routine fashion in Receiving Inspection, 
(a) the risk of having defective material in the delivered product is increased, (b) 
material detected as defective some time after receipt will not be as easily charged back 
to the vendor, (c) there will be no routine way to gather vendor quality history infor- 
mation "in order to initiate corrective action ... . . as indicated by the nature and 
the frequency of the nonconformance" (MIL-Q-9858, paragraph 3.525). The latter is true 
since the feedback of defect information from Design Engineers to a central vendor data 
collecting agency will be sparse. 


As to the position that the Design Engineer will inspect and test the material him- 
self and that Receiving Inspection will constitute a duplication of a if —oe 
Inspection checks the material, the Design Engineer's test will not be required, and Re- 
ceiving Inspection can probably do it ton less cost. 


However, there is good justification for the position that, in some instances, Re- 
ceiving Inspection will not possess the know-how or facilities to check certain parts or 
the cost of a suitable test jig may be disproportionate to the value of the test. ‘The 
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solution for this situation is for Receiving Inspection to delegate to the Designer the 
responsibility for the inspection of such an item of material and to provide him with 
forms on which to document the results of his examinations. This way there will be some 
assurance that the parts are actually inspected, and if the parts are defective, there 
will be a routine generation of failure information for corrective action purposes. 


The minimum criteria for Receiving Inspection in R&D are as follows: 


1. All contract material should go through Receiving Inspection. 

2. Material not intended for use in shippable equipment should be so marked, and 
then inspected and tested in accordance with the instructions of the material requisi- 
tioner. 

3. Material intended for deliverable equipment shall be receiving inspected and 
tested in accordance with specification requirements. 


st t 


Yet another interesting area in the application of Quality Control measures to R&D 
is the control of the accuracy of measuring and test equipment. A Design Engineer has 
said to me, "First an engineer loves his wife, and next he loves his test equipment, and 
I'm not sure this is always the order." Design Engineers are very sensitive about their 
test equipment. They are proud of their test equipment, the way a craftsman is proud of 
his tools. They will go to great lengths to accumlate the necessary test equipment 
which will facilitate their work. They are not receptive to the idea that anyone else 
will control their test equipment. 


In response to a reminder of the importance of having test equipment in a proper 
state of calibration before taking measurements for the establishment of specifications 
or for the purposes of engineering model product acceptance, they may reply: "Any good 
Design Engineer will make sure of the'state of his test equipment before starting a test." 
The implication is that since any good Design Engineer will make sure of the state of his 
equipment before using it for the above purposes, and since all Design Engineers are 
good, there is no need for a routine system of control. 


There are some special considerations which must be made with regard to control of 
maintenance and calibration of test equipment in R&D. In an R&D lab with an adequate 
supply of test equipment, certain pieces of test equipment may not be in continuous use, 
as they would be in Manufacturing, but may sit on the shelf until the time comes to make 
a particular kind of measurement on a new project. It is not necessary to adhere to the 
calibration schedule established for an item of test equipment when that item has not 
been used from the time of one scheduled calibration date to the next. Provisions should 
be made for this. If each item of test equipment were equipped with an "on-off" counter 
or a running time meter, it would be possible to establish a calibration schedule based 
on "times on" or “hours on" rather than elapsed calendar time periods. This might be the 
best kind of schedule. 


Another consideration is that certain items of test equipment may never be used for 
the purpose of establishing equipment specifications, or for purposes of engineering 
model product acceptance, but may only be used for trouble shooting, null indication, or 
as a reference during tests. Where the use of such equipment can be controlled, these 
factors should be considered in the establishment of the calibration schedule. 


The minimum criteria for the control of maintenance and calibration of electrical 
test equipment in an R&D laboratory should be: 


1. All items of test equipment in use in the laboratory should be uniquely identi- 
fied and recorded in a centralized file. 

2. The calibration period for each item of test equipment should be recorded in the 
file. 

3- ‘The calibration status of each item of equipment should be recorded in the file, 
and on the equipment. That is, file and equipment should show dates of last and next 
calibration. 

4. Test equipment beyond calibration dates should be marked to that effect. 

5. Specifications determined through the use of test equipment beyond calibration 
date, or acceptance test results taken on such equipment should not be accepted. 

6. Items of test equipment not in current use should be marked to that effect. At 
the time it is desired to take them out of such "temporary storage," they should be cal- 
ibrated and brought back into the system. 

7. All new purchased items of test equipment should be inspected and tested on re- 
ceipt, catalogued, and brought into the calibrations system. 
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8. Items of test equipment not used for measurement (that is items used for trouble 
shooting, null indication, or as a reference) should be marked to that effect and such 
use should be considered in the establishment of the calibration schedule. 


The two big factors in all of the above are the competence of the calibration and 
repair personnel, and the schedules established for equipment calibration. It goes with- 
out saying that the calibration and maintenance personnel must be competent. 


There are published schedules for the calibration of test equipment (one such sched- 
ule is as published in the Standards Laboratory Information Manual (SLIM) published by 
Bureau of Naval Weapons, Pomona, California). We must be careful not to waste money by 
checking equipment too often. Scheduled calibration periods should be based on having 
some small probability that an item of test equipment being calibrated will be out of 
calibration. Routine calibration should be scheduled so the probability of finding a 
piece of test equipment out of calibration is about five percent. 


Product Inspection 


Obviously, there must be a routine system of inspection for subassemblies, assem- 
blies and units immediately after (and possibly during) their assembly in the Model Shop, 
and also a final inspection prior to shipment. One area of inspection which is sometimes 
overlooked in an R&D) laboratory is the inspection which follows subsystem testing done 
by the Design Engineer. After a subassembly has been fabricated, it is turned over to a 
Design Engineer for tests. In the course of the Design Engineer's testing, modifications 
are made either by the Design Engineer or an engineering technician. In many instances, 
by intent, these modifications are not introduced in a workmanlike way. Resistors may 
be "tacked in"; leads may be "point-to-point"; long leads unsupported, etc. The reasons 
why modifications are introduced in an "unfinished manner" are valid. A lead being wired 
in, or a resistor being "tacked in" may very well be removed thereafter because the modi- 
fication being tried does not accomplish its purpose, or because of some other modification 
made later. Since work is being done in this fashion, it is therefore necessary that an 
inspection be performed after such design testing to insure that final changes are prop- 
erly installed. 


tractor Inspection Test 


One of the prime objectives in establishing a quality control system in R&D must be 
to keep from spending money foolishly. A very interesting consideration comes to our at- 
tention for this reason when we consider the inspecting and testing to be done by the 
contractor at the subcontractor. During the course of a subcontractor's designing, fab- 
ricating, inspecting, and testing a product for a contractor, contractor Design or Pro- 
ject Engineering personnel will have visited the subcontractor to give instructions, re- 
solve problems, and observe progress. If then, Quality Control personnel desire to visit 
subcontractors to inspect equipment and observe tests, does this not represent a dupli- 
cation of effort? 


The answer to this question is, "It depends upon who the Design or Project engineers 
are, and who the Quality Control personnel are." 


In an electronics R&D laboratory, some engineers get the reputation of being "good 
electrical men" and some get the reputation of being "good mechanical men." The happy 
combination of a "good electrical man" and a "good mechanical man" all-in-one is not al- 
ways present. In the words of one Design Engineer, "it takes more than one lifetime to 
produce this combination." This is why it is possible for an electronics Design Engi- 
neer to visit a subcontractor periodically, discuss the progress of electrical design, 
and still have an equipment containing an excess of workmanship defects delivered by the 
subcontractor to the contractor. 


There is no guarantee that sending a Quality Control representative to the subcon- 
tractor will preclude such an occurrence. What is important is that the proper mixture 
of electrical and mechanical backgrounds be blended in the Design, Project, and Quality 
Control personnel who visit vendors and subcontractors so that the subcontractors are 
not overwhelmed with contractor representatives and that all electrical and mechanical 
requirements for the equipment are met. This requires cooperation and common purpose 
between Project and Quality Control Supervision. 


Subsystem Testing 


What should be the role of Quality Control in the testing of engineering model sub- 
assemblies and units? Our response may be "Quality Control should test them all" if we 
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are quality control oriented, or "There is no need for Quality Control to test any of 
them" if we are not. 


It might seem appropriate to those of us in Quality Control that a Design Engineer, 
starting with a set of equipment input and output parameters, could design an equipment 
to satisfy specification requirements, calculate all internal parameter nominals and 
tolerances, and turn the first model over to Quality Control for "acceptance testing." 

It just does'nt happen this way. The Design Engineer must test at least the first model. 
The testing which the Design Engineer does on the first model as he receives it from the 
Model Shop is part of the design process. From the Design Engineers view the approach 
must be, of necessity, empirical. For this reason, what is Quality Control to do while 
the Design Engineer is thus checking out such first subassemblies, assemblies, and units? 
Where Quality Control will have the opportunity to observe or perform Systems Tests on 
the first engineering model, Quality Control acceptance testing of the first models, sub- 
assemblies, asseublies, and units is not too much of a concern. Where there are param 
eters specified by the system specification which i be observed at the systems level, 
but which mst be observed at the sub bly, ly, and unit level, it is appropriate 
that Quality Control personnel establish that the units meet such requirements. 


As a program progresses subassemblies, assemblies, and units for the second, third, 
and possible fourth engineering models go to Design Engineers for test. The intent of 
such testing becomes less experimental in nature and more for purposes of proof of com- 
pliance. Quality Control can now assume a larger role and so relieve the Design Engi- 
neer of the onerous, repetitive features of this work. 


Systems Testing 


Systems testing on engineering models is usually done by Project or Systems Engi- 
neers. Once more the question of Quality Control's role in Systems Test, and the avoid- 
ance of unnecessary costs must be examined. If Systems or Project Engineers are going to 
determine that the engineering model conforms to all its specification requirements, why 
should Quality Control be concerned with systems tests. Doesn't this represent unneces- 
sary cost? 


Systems tests can be considered to be made up of three phases, "debugging," proof 
of design," and "proof of satisfactory delivery condition." 


In "debugging" tests various elements of the system are modified as necessary to the 
point where they are compatible. There will probably be little Quality Control interest 
in this phase of systems test. 


In "proof of design" tests, Systems or Project personnel perform tests to determine 
that all specification requirements are satisfied. Depending upon the role of Quality 
Control in a given R&D laboratory, Quality Control may well have an interest in this 
phase of Systems test. 


In “proof of satisfactory delivery condition" tests, it must be demonstrated that 
the equipment is in a fit state for shipment. Even though it may have been proved that 
the equipment design is capable of meeting specification requirements, this may have been 
accompanied by much modification and rework. This makes it appropriate to give the sys- 
tem a final abbreviated operability test, or a test which approximates that which the 
customer will perform when he receives the equipment. This last phase of Systems Test 
Quality Control will be most interested in. 


Before giving my view of what the relative Systems Engineering/Quality Control Engi- 
neering relationship should be during R&D Systems Test, let us examine some of the moti- 
vations and viewpoints of the participants in Systems Test. 


During the course of design, construction, and Systems Test, the Systems or Project 
Engineer has been a party to the resolution of various problems, and to the establishment 
of various systems compromises. In many instances, these may have been verbal agreements. 
He is motivated by a desire to see the system pass the tests and be shipped on schedule. 
How does this influence his performance of the Systems Test? 


First, he is so intimately familiar with the system that he does not have to rely 
on the paper work which describes what the test requirements are and how the test is sup- 
posed to be conducted. He is not then aware that the course he is following is at vari- 
ance with the documented procedure as a result of all of the verbal agreements and ex- 
ceptions to which he has been a party. The result will be that when people later try to 
duplicate the test results following the written test procedure, they will not be able to 
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succeed. Second, there will be a natural tendency on the part of the Systems or Project 
Engineer to "pick a good day for the test," or to "tweak that extra pot" or to do some- r 
thing that will get the equipment over the hump when performance is marginal. : 


What then can Quality Control do? Quality Control can observe all the Systems Tests 
to make sure the test results are not being unduly influenced by a desire to meet ship- 
ment dates. Quality Control can run certain tests in accordance with the written test 
procedures in order to determine that the test procedures have been kept up to date. In 
a word, Quality Control can observe all tests, and conduct such other tests as it con- * 
siders necessary to assure that the equipment meets all its performance requirements. § 
One last interesting feature of the relationship of Systems, Project, and Quality Control 
Engineers during Systems Test involves trouble shooting which takes place in the event 


of failure. 
On the first engineering model trouble shooting is probably best done by Systems 
. and Project personnel. However, on subsequent models, it becomes appropriate for Quality 
a Control personnel to participate in trouble shooting, and on the last Engineering models 


to do the trouble shooting entirely. 


ESTABLISHING QUALITY CONTROL IN R&D eae 
fe es t: ‘or t trol in 


1. The ratio of people who can make policy in R&D to the total number of R&D per- 
sonnel is larger than the ratio of people who make policy in Manufacturing to the total 
number of Manufacturing personnel. Another way of saving this is that in R&D a smaller 
percentage of people follow detailed and routine instructions than do so in Manufacturing. 


The reason for this is that in an R&D) laboratory where many projects are underway 
simultaneously, a major responsibility for planning, innovating, and implementing each 
project resides with the people assigned to that project and not with their supervisors. 
In a real sense, designers, project engineers, and specification writers are policy 4 
makers. 


There are virtually many small companies under one roof, each of which plans, buys, 
fabricates, tests, and sells. The effect of this on the implementation of quality con- 
trol is that in R&D a relatively larger number of people must understand and agree with 
the quality control objectives and program than must do so in Manufacturing. In an R&D 
laboratory, virtually everyone has to understand and agree. It is mech easier to evade 
the quality control system in R&D. In Manufacturing on the other hand, there are rela- 
tively large numbers of people doing assembly and testing operations specified for them 
by fairly detailed instructions. These people do not have an equal opportunity or the 
inclination to evade the quality control system because the quality control system in 
Manufacturing has continuity and is very explicitly defined. 


2. The problem of "status" se the responsibility for Quality Control is 
greater in R&D) than it is in Marmmfacturing 


The practice in some R&D) labs is to appoint a Project Engineer to a project at the 
initiation of design effort. He is responsible for gathering bids, assigning tasks, is- 
suing funds, monitoring performance, meeting schedules, assuring specification compli- 
ance, and staying within budget. The Project Engineer "subcontracts" to various experts 
to design, fabricate, assemble, and test his product. 


‘ As an individual charged with overall project responsibility, there is a natural 
reason for the Project Engineer to believe it is his responsibility to establish the de- 
gree of quality control and the quality control standards on his project. In other words, 
there is anatural reason for the Project Engineer to believe Quality Control should be a 
service and not a control. The understanding which the Project Engineer has of the sys- 
tem requirements, and the first hand information which he has of what the customer wants, 

" and what compromises the customer would be willing to make between cost, delivery, and 

quality, gives the Project Engineer a natural reason for feeling his judgments should 

prevail in matters regarding quality control. 


This condition will present some difficulties if the requirements for Quality Con- 
trol organizational independence, as specified in MIL-Q-9858, are not properly imple- 
mented. 


3- The organizational relationships between customer, company (other than Quality 
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Control) and Quality Control are more complicated in R&D than in Manufacturing. 


There is an interesting set of relationships which develop between company Quality 
Control, Project Engineering, Design Eengineering,and Contracts, and customer Project 
Engineer, Contracts Officer, and Resident Military Inspector. 


There is usually a very good relationship between the customer Project Engineer and 
the company Project Engineer. Both usually have similar backgrounds and understand one 
another. The customer Project Engineer reports to a military R&D procurement branch and 
the company Project Engineer is part of the company's Engineering Department. 


As a program progresses, these individuals work together facing many problems and 
making decisions when they arise. When questions of marginal quality arise where either 
the schedule must slip, costs rise, or the specification requirements be relaxed, there 
is a natural tendency for the customer Project Engineer and the company Project Engineer 
to work out a "gentlemen's agreement" regarding future corrective action without going 
through the process of requesting a waiver to the specification through the company's 
Contract Administrator. A reason for this is that requesting a waiver takes time and 
may affect schedules, and some customer and company Project Engineers feel that the re- 
questing of a waiver reflects unfavorably on their performance. 


The advantage of such a course of action to a company'ts Engineering Department is 
that it can build up good will with the customer's Engineering Representative which is 
favorable to sales and general working relationships. The disadvantage to both parties 
is that either (or both) Project Engineers may leave the project and any "gentlemen's 
agreements" previously made, may be repudiated by his successor. 


The company Quality Control specialist's position is simple in this situation; 
"Make it like the specification or get a waiver." The Resident Military Quality Control 
specialist's position is the same. There is a temptation at this point for the contrac- 
ter and customer Quality Control representatives to team up in order to have the "gentle- 
men's agreement" aforementioned formalized. However, the Resident Military Quality Con- 
trol Representative reports to some military Marmfacturing branch and, in question of 
doubt, takes his lead from the military R&D Contracts Officer on the project. He may 
not therefore be in a position to insist that this be done. If the problem goes unre- 
solved into Manufacturing, the contract then being under the cognizance of a military 
Manufacturing procurement branch, the Resident Quality Control Representative may repu- 
diate any "gentlemen's agreements" made earlier in the program by the customer and com 
pany Project Engineers. The customer and contractor Project Engineers might take the 
attitude that they are not unduly concerned about the possibility of this at the time 
they make their agreement since, when and if this happens, they will probably no longer 
be associated with the project anyway. 


The above conditions tend to make allegiances develop which are associated with an 
individual's field of endeavor and not his company agency affiliation. This is wrong. 


RECOMMENDED APPROACHES 


Developing the Quality Control Program Plan 


Because a development laboratory is a location where many projects can be underway 
simultaneously, each at a separate stage of completion, and because each such project is 
essentially under its own management, it is very important that those managing such pro- 
jects be aware of the objectives, plans, responsibilities and authorities of Quality 
Control. 


For this reason, it is most appropriate that a Quality Control Program Plan be pre- 
pared which describes the objectives, approach, organization, responsibilities, and au- 
thorities of Quality Control. It is recommended that prior to the finalization of this 
plan it be circulated to key individuals for comment, criticism, and suggested improve- 
ments. 


In this regard, it is important to note that within the environment of an R&D lab- 
oratory there will be certain people who are looked upon as leaders because of their 
technical excellence, experience, or length of service with the company. The influence 
which such leaders exert may be out of proportion with that indicated by their position 
on the organization chart. It is most important that every effort be made to gain the 
support of such leaders and to tailor the Quality Control program so that its objectives 
and those of such individuals correspond. 
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It goes without saying that the manager of the R&D) laboratory must support and en- 
dorse the Quality Control program. In general it will be found that once Design Project 
Engineers, or those who provide engineering services understand the Quality Control pro- 
gram that has been agreed upon, they will cooperate. When they understand the proce- 
dures to be followed, the activities to be engaged, in the standards to be followed, and 
can utilize such knowledge in their planning, they will do so. They will not appreciate 
rejections based on standards not in existence when products were being designed, or 
admonitions for not following procedures that were not in existence when they were es- 
tablishing their schedule and their budgets. 


For this reason, it is recommended that in an R&D) laboratory, Quality Control be 
initiated on a project-by-project basis, beginning on each project at the bid stage. In 
this fashion, there will be agreement between Project and Quality Control personnel on 
the entire Quality Control program to be followed. Oe 


ESTABLISHING QUALITY CONTROL PROCEDURES 


Following the promulgation of the Quality Control Program Plan, the broad objections 
outlined in that document must of course be detailed in specific Quality Control proce- 
dures, and the Quality Control standards to be employed must be promulgated. Lou 


There are two approaches to the preparation of such procedures and standards, One 2 
approach is to prepare such material unilaterally and disseminate it to the laboratory ; 
as the final authority. Another approach is to distribute such procedures and standards : 
prior to issuance for criticism and comment, and to work toward common acceptance before 
finalization. The latter approach will be more time consuming but will lead to much 
more lasting results. 


Where an R&D Laboratory has a Procedures group, it is recommended that the services 
of this group be utilized to coordinate the finalization of Quality Control procedures 
which affect the activities of other groups. It is also recommended that such Quality 
Control procedures be included in the Laboratory's Operations Manual rather than in a 
separate Quality Control Manual. In this fashion, the Quality Control procedures which 
relate to the R&D) laboratory are under the aegis of the Engineering Lab Manager and there 
is no question about the relevance of such procedures to the Design and Project Engineers. 


The ty Control D: t 


In parallel with the formulation of plans and procedures, the Quality Control De- Bs 
partment must be staffed. hae 


In an electronics R&D laboratory, the primary skill which is required for a Quality 
Control. Engineer is competence in electronics. It would be helpful if he had a working 
acquaintance with statistics, and enough knowledge of mechanical engineering and shop 
practices to know where to go to get answers. The Quality Control Engineer mst be able + 
te recognize good and bad workmanship in all of the forms it takes in electronics equip- ; 
ment. Personality requirements are equally severe and individuals of the type described 
above must be developed from personnel selected for potential capabilities in the re- 

} te areas. A possible approach to organizing an R&D) Quality Control Department is as 
‘olLows 


A Project Quality Control Engineer can be assigned (on a part time basis for small 
projects) from the Quality Control Department to each job. This individual can be given 
the responsibility to plan an effective Quality Control program on his project. This 
would include preparing test procedures for parts, units, and systems under various en- 
vironmental conditions, participating in the disposition of rejected material, and in all 
other ways, acting as "Quality Centrol Manager" on that job. 


This individual can have the Quality Control services of inspectors, testers, test 
equipment maintenance and calibration personnel, and environmental-testing personnel made 
available to him. 


the assignment of the Quality Control responsibility, the Project Quality 
Control Engineer will,in a sense, become part of the project team and become a valuable 
aid to the Project Engineer. 


It is most important that the proper attitude exist between the Project Quality Con- 
trol Engineer and the Project Engineer. The Project Quality Control Engineer can be of 
great help to the Project Engineer in meeting the objective of shipping, on time, equip- 
ment which complies with specifications. The Project Engineer can be of great assistance 
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to the Project Quality Control Engineer in supporting his activities and his decisions. 


As was mentioned above, the consideration of who is responsible for quality in an 
R&D laboratory is just as important as the consideration of who is responsible for quality 
in Manufacturing. 


In an R&D laboratory, the Project Engineer is given the responsibility for design- 
ing a new product and having a model fabricated and tested. He is responsible for the 
shipment of a product which conforms with specification requirements in accordance with 
a schedule. He must stay within a budget. “e is given the authority to buy services, 
control expenditures, and deal with the customer. When in the implementation of a Qual- 
ity Control program in R&D, someone other than the Project Engineer is given, or assumes, 
some responsibility for quality a natural response of a Project Engineer might be that 
the Management is taking away part of his authority; therefore, he may conclude he can 
no longer be held completely accountable to control budgets, schedules, and specification 
compliance. 


To a certain extent,this is trueshowever,the response of one Manager of Engineering 
to this contention is interesting. To the above he replied, "I have always had the ul- 
timate responsibility for the quality of the equipment shipped from this laboratory. I 
am not taking away the responsibility of the Project Engineer for qualtiy. I an dele- 
gating to Quality Control part of the authority for control of quality I have always 
had." 


Regardless of who is responsible, the fact remains that where the Project Engineer 
and the Project Quality Control Engineer have a common desire to work together for the 
mutual interest of the company, with good faith and mutual respect and understanding, 
they can both help one another considerably. 


. 


CONTROLLING QUALITY IN IHE PRODUCTION-ENGINEZRING CLIMATE * 


J.» Braverman 
Chief, Quality Control Section 
Production Engineering Br., Operations Div. 
Frankford arsenal 


Introduction 


The control of product and process quality in the production-engineering climate 
where new and modified weapons, ammunition, and fire control instruments are mam- 
factured in diversified quantities and types is a real challenge. This challenge is 
surrounded by quality problems where the solutions do not necessarily commence with a 
rigid search for answers as mech as it begins with the flexibility of your percep- 
tions. The real task, and it is generally a tailor-made one, is to place the problem 
in the right perspective and to measure it against the conditions of the actual indus- 
trial situation in which it originated. Thus, in this situation, with independence 
of movement and direction, quality control can objectively approach the wide variety 
of daily quality problems indigenews to the production-engineering climate and pro- 
vide rational conclusions. The objective approach, however, mst possess an organized 
method of attack. A systematic procedure to controlling quality is a requisite. 


Purpose 


It is the intent of this paper to describe the methods ami techniques employed 
at the U. S. Army Ordnance installation, the Frankford arsenal, in "Controlling Qual- 
ity In The production-Engineering Climate." Frankford Arsenal has a mamfacturing 
capability for production of development type items and limited, initial production 
runs. This is a means whereby the necessary production engineering is performed dur- 
ing the phase of the development of an ordnance item and the phase where private 
industry undertakes quantity production using product engineering drawings and speci- 
fications which delineate the desired requirements. 


Background 


Before entering into this discussion, however, it could prove advisable to the 
reader if a brief description was presented as to the Frankford Arsenal mission 
in light of today's industry--yovernment teamwork. This can serve as a denominator 
over which the direction and activities of quality control can be measured or evalu- 
ated. 


The Frankford Arsenal today is primarily a research and engineering center with 
responsibility to the nation for developing and engineering for mass production, new 
weapons, ammunition, materials, and methods. Frankford arsenal shops are recognized 
authorities in the United States for optical, electronic, mechanicai, hydraulic, and 
propellants production engineering. The optical shops produce optical ranging, sight- 
ing, and observation equipment, and afford assembly facilities for pilot and production 
runs of complex optical instruments. ‘he electronic shops have facilities for devel- 
oping and producing pilot lots of printed circuits, microminiaturized components, and 
potted parts for computing, fire control and integrated weapon systems. Complete 
facilities for producing pilot runs of ammnition ranging from cal. .30 rounds to 
155mm shells, and mechanical time fuzes are maintained in the ammnition and fuze 
shops respectively. ‘These, together with a variety of auxiliary shops, are capable of 
producing complete, integrated guidance and control portions of weapon and missile 
systems, small arms and major caliber ammnition, metai components, and propellants. 
These shops are equipped with fine multi-purpose tools, the most modern equipment, 
special and automatic tools only required to develop and test out important new mam- 
facturing techniques, and manned by highly trained career workmen, technicians, and 
scientists. 


* ASQC Literature Classification System - 300:70;419 
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In the case of this presentation, the Frankford arsenal shops are under the Manu- 
facturing Branch which is part of the Operations Division. ‘he Operations Division 
also consists of a process Inspection Office, Production Engineering Branch (the 
Quality Control Section is a segment of this branch) and Program Management Branch. 
The Mamfacturing Branch is made up of approximately 11:00 skilled machinists, tool- 
makers, gage makers, and machine operators assigned throughout 30 shop buildings. 


Discussion 


In this discussion, an attempt will be made to explain the responsibilities and 
techniques associated with the Quality Control Section during the various stages of 
product growth. Quality Control, as described in this paper, is a mamfactuwring sup- 
port activity, concerned with detection, verification, correction, and prevention 
aimed at establishing and maintaining optimum levels of product and process quality. 


The details of the quality control function in the Operations Division include; 


1. Plan, direct and coordinate quality control activities throughout the entire 
Operations Division (includes Manufacturing, process Inspection, Program 
Management, and production Engineering). 


2. Develop and verify quality control standards, plans, and procedures for div- 
ision-wide application. 


3. Contimously analyze, evaluate and report product and process quality and 
cost of achieving and maintaining this quality. 


4. Investigate, analyze and correct product and process quality problems, and 
initiate control measures to prevent reoccurrence, recommending corrective 
action. 


5. perform research and development in sampling plans, quality control and reli- 
ability techniques and methods. 


6. Provide training in quality control. 
7. Delineate quality control requirements to suppliers. 


8. Feed back product and process quality data for utilization in design and 
production. 


9. Investigate and develop new methodology in inspecting and measuring. 
10. provide technical guidance and assistance in the field of quality control. 


It has been found that a three-stage approach to controlling product and process 
quality is most effective for our production-engineered jobs. ‘These stages are consid- 
ered as; 


STAGE I - PRE-PRODUCTION 
STAGE II - INITIAL PRODUCTION 
STAGE III - PROCESS STABILIZING AND REG'LAR PRODUCTION 


Inherent in each of these stages are the various quality control techniqes that will 
be described. Figure 1 depicts these techniques. It should be mentioned that while 
the techniques have been tailor-made to fill the quality control needs at this instal- 
lation, it is felt that they are general enough to be possibly considered for other 
types of production-engineering activities. 


STAGE I - PRE-PRODUCTION 


Preproduction represents that stage in the mamfacturing cycle preceding the 
actual mam facturing, where the procurement, methodizing and planning are performed. 
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It is in this stage where quality control is concerned with the development of inspec- 
tion plans for process inspection, the review of past mortality on like or similar 
parts, the development of new or modified sampling plans, the investigation of new or 
refined gaging and/or control devices, the development of quality control requirements 
for procured material, and the review of drawings and specifications for design, prod- 
uct, or process deficiencies, as well as quality control needs in subsequent stages 
of product growth. Details of these factors are explained below. 


1. Development of petail Inspection Plans for Inspection 


Detail Inspection Plans are developed to assure standardization and proper 
degree of inspection during processing and assembly. Included in such plans are the 
characteristics to be checked, classification of such characteristics, acceptable 
quality levels, types of measuring equipment and special instructions to inspectors. 
Inspection and manufacturing personnel normally receive these plans prior to Stage II, 
when the job commences in manufacture. Where the job is of a small quantity (1-25 
pieces) and of short duration (less than a few months), minimm effort is directed 
towards the development of such inspection plans. An example of the Inspection Detail 
Plan is presented in Figure 2. 


2. Quality alerting Procedure 


Inherent in any job shop is the processing of jobs (components or assemblies) 
that possess some degree of similarity to jobs that have been previously produced. 
the Quality alerting Procedure is a technique that includes the review of past qual- 
ity performance of similar jobs. As a result of this review, past repetitive quality 
problems or trends are highlighted for the attention of methods engineering, mamfac- 
turing, or process inspection. The "Quality Alert Report" can accompany the job 
through all stages of mamfacture, alerting mamfacturing and inspection supervisors, 
mach:inists, operators and inspectors as to past problems on the job or a similar one. 
The value of the Quality Alerting Procedure is that while the time interval between 
like or similar jobs can be a few years, past objective evidence is excellent refer- 
ence information. While our memories can piay havoc with us in the production- 
engineering climate due to the wide variety of jobs being handled concurrently, 
objective evidence fills the need. Figure 3 is an example of the "Quality Alert 
Report." 


3. Development of Sampling Plans 


Statistics is the science of working with data noticeably influenced by chance 
causes, Whether we have data from mass production or from short runs, as long as we 
have some degree of repetitiveness during the production cycle, sampling developed by 
Statistical methods can be applied. (In 1920, the development of the theory of small 
Samples permitted the departure from the need of masses of data). AS most quality 
control people know the basis for the statistical quality control approach is the con- 
cept of alikeness, rather than quantity. Alikeness implies that whether you produce 
25 pieces or 2500 pieces, the amount that one part differs from the preceding one is 
not significant. In general, tailor-made sampling plans, with their risks and limita- 
tions understood, can be applied to determine the degree of alikeness. As a matter of 
fact, due to the heterogeneous variety and quantity of work flow under job lot condi- 
tions at this installation, tailor-made sampling plans are the routine rather than the 
exception. Variations of attribute (single, double, mltiple, and continuous) and 
variable sampling are thus employed. ’ 


The AQL (acceptable quality level), AOQL (average outgoing quality limit), and 
LTPD (lot tolerance percent defective) bench marks are considered when constructing 
Sampling plans. Our approach to the application of sampling plans is that it should 
guarantee customer requirements, it should be easy to understand and apply, it should 
be statistically sound, and it should be surveilled when in operation. An example of 
a specialized attribute sampling plan for small lot sizes is illystrated in Figure lL. 
A further example which depicts characteristics classified and their. associated ACL's 
is shown in Figure ha. In figure lb is an example of a Multi-Level Contimous attri- 
tate Sampling Plan. 
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SINGLE SAMPLING TABLES FOR SMALL LOTS 
(1 TO 200 PCS.) AND A.Q.L.’s LESS THAN 0.65% 


MAJOR MINOR 
Lor size | SAMPLE | acc.| rev. | SAMPLE | acc.| rev. 
1-25 100% 100% 
26-45 25 1 15 
61-75 30 o| 1 15 1 
76-110 30 o | 15 1 


NOTES: 
|. MAJOR CHARACTERISTIC 

A MAJOR CHARACTERISTIC IS A CHARACTERISTIC WHICH IS A 
REQUIREMENT ON THE FINISHED PRODUCT, A CHARACTERISTIC 
WHICH COULD PREVENT THE COMPONENT OR ASSEMBLY FROM 
MEETING THE FINISHED PRODUCT REQUIREMENT AFTER PROCESSING 
OR A CHARACTERISTIC WHICH COULD AFFECT THE ASSEMBLY OF 
THE END PRODUCT, OR CAUSE FAILURE TO FUNCTION IN 
ACCORDANCE WITH THE INTENT OF THE DESIGN. 


2. MINOR CHARACTERISTIC 
A MINOR CHARACTERISTIC IS A CHARACTERISTIC WHICH WHEN 
DEFECTIVE, DOES NOT MATERIALLY REDUCE THE USABILITY OF 
THE UNIT OF PRODUCT FOR ITS INTENDED PURPOSE. SUCH 
CHARACTERISTICS ARE RADII, CHAMFERS, CLEARANCES, FRACTIONAL 
TOLERANCES OR EQUIVALENT, WORKMANSHIP, PIECE MARKING, AND 

ROUGHING OPERATIONS. 


Fiag.4 


473 


- : ; 
. 
“a 
. 
| 
“| 
Ye 
: 
. 


1961 ASQC CONVENTION TRANSACTIONS 


474 


40140 


6 


SaQwasso-gng (02400 
ONIN | | | | | ' vw | | worwn | | 


bu 

$2.90 

O% 420g 40 


162.10 

wg 

49 


IWIILd0 


69 
ON 440g JO OP 

40 
“Puno wy 
Se 
$Saups0H 6u,005 er 
64,009 
ONO 


sub) 


busy 


“SMOONIM 


uiw O1 abun, jo, 
buy @ abuoy 
ON 440d a@2uasaig ie 
40 
Se 
SSAUPIOH 409% 


ie 
uiw oO, 
Gn abuos 
Puce O10 % abuos 
puog 40 
40/pu0 juawad 
busy puod 
0, puadsag 
Puo 10443 | 


$3903M ‘SHOMMIN 


| 
| 
| 
| 


0400 4242 LIV de 
ON 440g 40 a2uasaig 
4a,wOud 

(01-02 ) 


( O1- 02 % ON) 
| 
worun 


$3791138 


wiy yO 
ON 440g 40 a2uasaig 
sapwoud Sr 

$$aups0H Er 


(01-02 ) 


(01-02 % ON) 
ardoyans 
suawad 
(0904 420g 
yibuar 10904 andy 
(0900 
2 
$3SN37 


S3ITGW3SSV-8NS ONV SLN3W313 WWOlldO S3dAL SNOINVA 


104 


NOILO3dSNI NI G3SN 38 OL GNV S193430 4O 


Fig. 4a 


| 
| i 
| | 7 
| | 
| 
| 
| 
| 
| 


PHILADELPHIA, PENNSYLVANIA 


FLOW PROCESS CHART FOR MULTI-LEVEL 
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l. Investigation of New and Re*‘ned Gaging and Measuring Devices 


The production-engineering of diversified quantities of complex and less complex 
items necessitates the need for a flexible gaging measuring program. Coupled with 
this, from the quality control viewpoint is the search for adaptations of such gaging 
and measuring equipment to semi-automated and automated systems for highly efficient 
process control. Quality Control is not concerned with specifying and ordering gages 
and measuring equipment--this is the job of process inspection. Quality Control is, 
however, most concerned with advancing the application of new or refined gages and 
measuring equipment to realize increased inspection productivity, repeatability of 
readings, greater accuracy on tolerances of the magnitude of .00005 inches, and means 
for rapid control of process quality whereby machines or operators can control and 
guide their process before spoilage is realized. For example, investigation can be 
undertaken in an effort to adapt pneumatic or electronic gaging equipment to com- 
puters so that the capability of processes can be rapidly determined. Aan extension 
of this idea in variable control charting is where the subgroup individual measure- 
ments are fed directly into computers together with control limits. The rapid answer 
indicates whether to contime to run the process, adjust process, or stop process. 
Another example of this type of investigation can be the attempt to automatically 
perform visual inspection on brass ammunition cases on a contimous basis. An 
electro-mechanical device with an automatic loading and feeding device can be consid- 
ered for evaluating surface nicks, dents, etc., with the findings fed into a computer 
which automatically calculates the total fraction defective rate and fraction defec- 
tive for Major and Minor defects, 


Then again, consideration might be given toward a gaging device which will 
standardize upon optical inspection, to eliminate the possible wide influence of 
human judgment. The light source, positioning of the elements, viewing, and compari- 
son with standards would be made uniform for all elements. 


There is a very positive need in quality control, early in the life of the 
product, to consider newer and better ways of gaging am measuring. 


5. Quality Requirements for Suppliers 


Difficulties by suppliers in producing satisfactory ordnance quality can be re- 
lated many times to inadequacies in supplier's quality control, competitive forces, 
and lack of understanding on the part of the supplier and the Ordnance as to what con- 
stitutes satisfactory product quality requirements. These problem areas create the 
important need for Ordnance to better analyze its product requirements and develop 
more meaningful specifications, to provide suppliers with realistic quality assurance 
provisions, to maintain real cooperative relations with suppliers, and to evaluate 
the supplier's product and process quality more accurately and objectively. 


Thus, in helping the supplier to effectively distribute his control efforts in 
accordance with Frankford arsenal product quality requirements, we are now deline- 
ating to our suppliers, at the time the bids are asked for, general and detailed 
quality assurance provisions, General provisions (Figure 5) include such features as 
Supplier (Contractor) Inspection System, Verification of Supplier (Contractor) Inspec- 
tion System, Tolerances, Workmanship, Finishes, etc., while detailed provisions 
(Figure 6) spell out the AQL's (acceptable quality levels), CC's (classification of 
characteristics), and sampling plans. We feel that good commnications between 
Frankford arsenal and its suppliers (contractors) is the key to good customer rela- 
tions, avoiding trouble, and getting out of trouble as soon as possible. With 
effective communication of product quality requirements, suppliers will know what is 
expected of them quality-wise, possess a good idea of quality control techniques 
desired by the Frankford arsenal, be assured of consistent practices between supplier 
inspection and Frankford arsenal inspection surveillance, and effectively provide 
sound, objective evidence for Frankford arsenal evaluation in product acceptance. 
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A. Product Quality Requirements 


Insulation Resistance, with all switches closed. The insulation resistance 
n the Owing terminals: 
shall be 100 megohms minimm, when measured at 500 ¢ 10 volts, DC. 


Circuit Resistance. The circuit resistance of the 3 switches and all combina- 
tions of 2 of the 3 switches through each closed circuit shall not exceed 0.2 
ohms maximum under the following conditions; open circuit voltage 5.5 ¢ 0.5 
volts, DC; current 100 ¢ 2 milliamperes (Switches 1 and 2 closed, 1 and 3 closed, 
and 2 and 3 closed. Circuits: 1-8, 2-7, 3-5 and 4-6). 


Jolt. The switch and cable assembly shall withstand the jolt when tested in 
accordance with Mil-Std-350 when assembled in a test vehicle similar to the 
Arm Safe Device, here shall be no completed circuit ar closed switches upon 
completion of the test. 


Jumble. The switch and cable assembly shall withstand the jumble when tested 
In accordance with Mil-Std-351. The assembly should jumble in a container when 
assembled in a test vehicle similar to the arm Safe Device. There shall be no 
completed circuit or closed switches upon campletion of the test. 


Forty (uo Foot prape The switch and cable assembly shall withstand the drop 
en tes accordance with mil-Std-352 when assembled in a test vehicle 
Similar to the Arm Safe Device. There shall be no campleted circuit or closed 

switches upon completion of the test. 


zive_(6) [ect Deep. The switch and cable assembly shall withstand the drop 

en tes accordance with Mil-Std-356, when assembled in a test vehicle 
Similar to thearm Safe Device. ‘There shall be no campleted circuit or closed 
switches upon completion of the test. The assembly shall be wperable after this 
test. 


Vibration. The switch and cable assembly shall withstand the vibration when 
tested In accordance with Specification Mil-E-005272a, Procedure XI. after 
vibration the assembly shall be operable. 


Low Temperature. The switch and cable assembly shall withstand the low temper- 
ature (-LO°F) when tested in accordance with Specification mMil-B-005272c, 
Procedure I. The assembly shall be operable at this low temperature. 


High Temperature. The switch and cable assembly shall withstand the high 
125°F) when tested in accordance with Specification Mil-E-005272c, 
Procedure II. The assembly shall be operable at this high temperature. 


Temperature Shock. The switch and cable assembly shal] withstand temperature 
ocks when tested in accordance with Specification mMil-£-005272C, Procedure I. 
Upon completion of temperature conditioning, the assembly shall be operable. 


Be Quality assurance provisions 


Contractor Inspection. Unless otherwise specified, the contractor or supplier 
Shall be responsible for the performance of all inspection requirements prior 
to submission for Government inspection and acceptance. Except as otheywise 
specified, the contractor or supplier may utilize his own facilities or any com- 
mercial laboratory acceptable to the Government. Inspection records of the 
examinations and tests shall be kept complete and available to the Government. 


Government Verification for Contractor Inspection. 


ae All inspection operations performed by the contractor or supplier and all 
material supplied by the contractor or supplier will be subject to 
Government verification. Verification will consist of: (1) Government 
Sampling to measure quality of product, (2) check of the contractor's or 
supplier's inspection operations, and/cr (3) periodic analysis of the 
contractor's or supplier's inspection and test records Fi iy 
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6. Review of Drawings and Specifications to Determine Potential Trouble Spots 
Before Job Starts 


Detection and prevention are as synonymous with the quality control as accept- 
ance is with inspection. ‘he before-the-fact prevention is certainly more economical 
and dramatic than a post-mortem approach. ‘hus, quality control reviews the drawings 
and specifivations of new jobs for the purpose of recognizing potential quality prob- 
lems, design deficiencies, or methods engineering conditions which might be adverse 
to high product yield. The recognition of such problems or deficiencies are brought 
to the attention of the proper personnel in a written format as depicted in Figure 7. 
This does not emi here, inasmch as there is a follow-up on the corrective action 
which was to be taken. You will note on the bottom portion of the report that the 
type of corrective action to be taken is required. 


STAGE II - INITIAL PRODUCTION RUN 


The Initial Production Run Stage represents that portion of the initial mam fac- 
turing cycle up to approximately 10% to 25% completion of the order. Here-.is where 
the effects of the methods, men, material and facilities upon the quality, quantity, 
and economic requirements are closely scrutinized, and altered where necessary. It is 
at this stage that the effects mst be particularly in consonance with the require- 
ments, for what follows is only as good as what has been initially established. 
Quality-wise, the costs of detection and prevention in this stage are only a fraction 
of how extensive the costs would pyramid to if the undesirable effects were permitted 
to prevail during Stage III, or the Regular Production Stage. The quality control 
activities in Stage II, which are described below, include reporting method, attri- 
bute control charting, process capability studies, instruction to inspectors, audit of 
quality, training, development of new techniques and quality control and inspection 
teamwork. 


1. Product & Process Quality Reporting Method 


Since the ultimate measure of quality in process is the actual evidence of the 
product yield and mortality, provisions mst be made for establishing a method of 
maintaining and presenting the “quality score." How should the data be submitted by 
inspection? How should the data be collected, analyzed, and presented? How fre- 
quently should reports be issued? How sensitive should be the reports? ‘These 
questions, which are usually uniqe to the production-engineered type of job, con- 
front our quality control activity. 


An effective method of reporting product and process quality in the production- 
engineering climate includes; 


a. Inspection Reports (Figure 8) are generated by inspection for all types 
of components and assemblies at all stages of product growth. ‘This report is carbon- 
interleaved, consisting of six copies. Copies are distributed to the mam facturing 
supervision, methods engineering, production control, cost control, quality control 
and process inspection. 


b. ‘the daily feedback of data from the Inspection Report is collated, 
analyzed, and reported by quality control in the following manner; 


sey qe Summary Report - describes percent defective rates on 
weekly and to-da or components associated with the various jobs (Figure 9). 


Weekly Re of Leading Defects - points out leading defects and trends 
of various components abe Subassemblies (Figure 10). 


Monthly Product and Process patty i - graphic presentation on a 
percent defective basis, the total mor y, Scrap and rework (on a monthly basis) 
charged to the appropriate mamfacturing shop. Control limits (f¥ 4 27) are incor- 
porated in this report (Figure 11) so that performance can be compared to a standard. 


Qut-of-control conditions are associated with the leading defect(s) so that correc- 
tive action can be directed most effectively. 
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To: CHIEF, 
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WEEKLY QUALITY SUMMARY 
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There are additional aspects of the reporting system in which brief reference 
will be made. These have proved important in the area of presenting contiming evi- 
dence of the actual product quality. Such reports are; 


First piece or Layout Inspection Report Used to record results of 
first piece inspection during processing and/or layout inspection on new or modified 


castings Figure 12. 


Frequency Distribution Report - Where inspection readings are made in 
variables and the Information 1s to be recorded, use is made of the Frequency Distri- 
bution Report Figure 13. This report can be attached to the Inspection Report or be 
embodied in with a report of product or process quality which is fed back to methods 
or design engineers. analysis of the frequency distribution is accomplished by 
quality control. We look upon the frequency distribution as a tool, as well as a con- 
cept. This is an effective way to condense and summarize contimous data and permit 
the reader to quickly form an idea as to what the central tendency and variability 
approximate. 


2. attribute Control Charts ("p"and "c" charts per product and shop) 


The attribute control charts ("p" and "c") charts, which are concerned with 
judging the stability of the fraction defective and mumber of defects generated in 
the process, are good tools that can assist in identifying quality problem areas. 
Using these charts, a shop, process or product can be justifiably considered stable 
or unstable, The installation of such charts is either based upon past data about 
the particular job or a similar one, or assuming what the sample size and fraction 
defective can logically approximate. In Figure 1) is an example of "p" chart em- 
ployed on a new job. 


3. process Capability Studies 


Process capability studies find wide use during the "Initial Production Run 
Stage" as well as during the "process Stabilizing and Regular production Stage." 
Failures to meet reasonable specifications result from (1) inadequate process capa- 
bilities and/or (2) inadequate operating controls. Only the knowledge of the existing 
process capability can help in diagnosing the cause or causes of the failure. Know- 
ledge of machine capabilities and control allowance requirements avoids the 
dissatisfactions which result from inability to meet specifications and the excess 
costs involved in over tooling. The idea of having identical sized parts coming from 
a process, whether of short run or long duration, is certainly false. In actuality, 
it is customary to find a central tendency with larger and smaller scattering on both 
sides of the central tendency. 


The application of process capability concepts in the grotnetien-engiamaing 
climate is not limited by the generally irregular production or r of ¢ ts 
and assemblies. ue to the nature of the smaller quantities on our order, enaller 
Sample sizes are used in conjunction with capability studies. Sample sizes vary be- 
tween 50 to 100 pieces. Naturally we lose a certain degree of reliability of our 
interpretations as the sample data decreases. (Data on averages become more reliable 
as the square root of the sample size, while data on spread become more reliable as 
the square root of twice the sample size). However, for expediency sake, coupled 
with our willingness to risk a certain decrease in accuracy of our answer, we employ 
capability studies using 50 to 100 pieces on potential trouble spots before the fact, 
or leading defects during the fact. The important thing is that the statistical 
analysis on small samples still gives the best estimate possible of the process and 
product when contrasted to a practical decision. An example of such a capability 
etudy is presented in Figure 13. 


4. Quality Control Instructions to process Inspection 


The carrying out of a quality control technique, method, or plan involves in 
many cases, assistance and participation by process Inspection. Written instructions 
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must be made available to the process inspectors to prevent misunderstandings, 
oversights and the like. This approach affords accurate communication and is of par- 
ticular importance in the production-engineering climate, because many times you 
don't get a second chance to run the job or find that a second lot will be processed, 
An example of a "Quality Control Instruction" is given in Figure 15. Such instruc- 
tions can include variable control charting instructions, procedures for carrying out 
a@ special test, the change in sampling frequency, the initiation of a process capabil- 
ity study, or the application of a new piece of test equipment. 


5. Audit product & process Quality Results & Initiate Corrective Action 


Auditing of product and process quality results is usually the forerunner of 
identifying and solving our significant quality problems, Significant, in the 
production-engineering case, implies out-of-control mortality, heavy dollar losses, 
noticeable interruptions in processing, functional shortcomings, inspection inaccur- 
acies, unrealistic specifications, etc. Auditing includes daily shop visits to 
observe attribute and variable control chart results; results of special tests: re- 
view of daily, weekly and monthly quality trends and costs; and evaluation of 
incoming and outgoing qality. Corrective action on problem cases in the production- 
engineering climate usually means a team approach. The team brings together a small 
number of people, each of whom has something different to contribute to the solution 
of the problem, The nature of the problems can touch upon methodizing, machine, men, 
material, inspection, gaging, tolerancing and the like. ‘The corrective measures in 
process and product control is frequently elusive. While there are basic statistical 
tools (capability study, control chart, regression study, significance tests) that can 
be used by quality control, there are times when getting an answer still appears 
futile. It is here that our team approach proves of immeasurable value. 


6. Training in Quality Control Techniques 


when training in quality control methods and techniques is provided to process 
inspectors, methods engineers, and supervision, their background and place in the 
organization is considered, as well as the needs to which the training will be ap- 
plied. The training attempts to provide simple concepts or fill a particular need 
during the class meeting, which average 2 hours per meeting. Ample handouts and 
demonstrations accompany each class session. Regardless of the group of individuals 
being exposed to the training, interest mst be aroused in order to take the statis- 
tical quality control concepts and application away from the "mystic" stage and into 
the useful tool stage. Actually we look upon training from the standpoint of a 
“problem-Solving Clinic." 


Simple demonstration devices are used wherever possible. Little or no homework 
is required, since on-the-job application can indicate the degree of proficiency. 


Subject matter covered includes; 


Frequency distritution study 
Variable control charting 
Attribute control charting 
Attribute sampling 

Variable sampling 

Statistics made easy 

Accuracy in reporting 

Inspection short-cuts 

Gaging accuracy 
Inspection-production relationships 


we feel that training in quality control methods and techniques is a contiming thing. 
Personnel wrnover, greater amplification of basic fundamentals, newer ideas, etc. 
necessitates contimous training. An example of a training handout is presented in 
Figure 16, 
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FROM: Ind Qrp 
Operations Div. VA C ONTROL 


pace 2 of 2 


Sumusct: Statistical Method of om, of 
id Cartri WA Blank . m™ t 
Blank’ Leading : 


Purpose: To establish Quality Control limits for weight of powder charge at 
the operation mentioned in the subject. 


REFERENCE Inspection data taken frax August through November 1960 


FORMATION OR 


1. 9520: Take a sample of five cartridges from each loading machine in opera- 
tion at least every thirty minutes, and more frequently, if necessary, Inspect 
for visual defects, remove wad, and weigh, using a chain-o-matic balance, powder 
charge to the nearest .01 grain. Recordwights, compute averages and extreme varie- 
tions and plet om control charts. The following quality control limits shall apply: 


2. 8833 and 9520; Observe the following limits; 


Limits for Avg. limits for E.V. 
of Samples of 5 or Samples of 5 
Weight of powder 18.20 - 18.80 grains -50 grains 


3. 9520: When visual defects are encountered or when an average or extreme 
variation falls outside the prescribed control limits, shut down the machine for 
adjustment. Subsequent to adjustment, recheck weights and, if satisfactory, permit 
be machine to operate; otherwise, shutdown the machine again for additional 

ustment. 


4. 8833: Make corrections and adjustments when requested by 9520. 
5. 9650: Will supply necessary forms and charts, 


AJWard/mp/21254 

TRI BUTI ON: 

8800 - Mr. Penn, 212-1 9650 - Mr. Braverman, 228-3 
9500 - Mr. Furmansiki, 228-1 9650 - Mr. Ward, 228-3 
9600 - Mr. Hewitt, 226-3 9520 = 212-1 (3 

9600 - Mr. Gatter, 226-3 9650 = 226-3 


9621 - Mr. Walters, 226-3 
9520 = Mr. O'Neill, 212-1 
8833 - Mr. F 209-3 (2) 


Approved: 


et Quality Control Seet (9650) 
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| Prod, Eng'rg. Br. (9600) 
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7. Development of New Techniques for analysis of Quality Problems 


The implication at this point of developing new techniques for attacking and 
analyzing quality problems is perhaps better described in the familiar phrase, "there 
is a better way to do it - find it." There is no question that often the statistical 
and engineering man-hours involved in handling a quality problem becomes quite time 
consuming, laborous, and costly, and does not provide the answers in sufficient time 
for the industrial situation. This need, particularly in our type of manufacturing 
situation, has forced quality control to explore short and rapid statistical tech- 
niques, nomographs, tables, and charts as an adjunct to the more sophisticated 
statistical techniques. In actuality, we find that a dichotomous approach in the use 
of statistical techniques in the production-engineering climate is the best. he more 
accurate, detailed tests and the short and rapid tests are both considered, with the 
appropriate type of test applied with the need. Naturally, with the short and rapid 
tests, we realize a decrease in efficiency of our answer. Basically, it is a matter 
of balancing the decrease in accuracy of one answer with the increase in precision, 
time, and manpower, In Table [ is listed a comparison of some detailed and rapid 
tests. 


8. Coordinate Quality Control & Process Inspection Effort 


The coordination of quality control and process inspection activities is essen- 
tial during the Initial Production Run stage. This is necessary in view of the 
variety of quality control techniques employed and problems recognized by process 
inspection. An example of this coordination is depicted in Figure 17. Coordination 
of effort is of importance in thie stage since many new ideas and techniques are being 
tested for feasibility of application during Stage III. 


STAGE III - PROCESS STASILIZING & REG'LAR PRODICTION 


In Stage III, there is a general stabilizing of quality effort. Some settling- 
down or elimination of significant defects occurs, Naturally stabilizing refers to 
all those contributions of the process which can contribute to product or process 
quality shortcomings. This stage contimes through to the completion of the order, 


1. Comprehensive Management Report of Product & Process Quality 


Comprehensive reporting in Stage III is concerned with keeping management 
informed as to the product quality, types of leading defects, out-of-control condi- 
tions, losses due to scrap, and corrective measures taken to correct quality 
discrepancies, Reports to and from management regarding product and process quality 
also clarify the problems that mst be solved, 


An important fact is considered in our approach to reporting. We strive for clear, 
concise reports. It should be pointed out that in the production-engineering type cli- 
mate, with its wide variety of items of low and medium density, it is often easy to 
develop reports which "snow-ball" in size to accommodate for the many components, and 
processes, AS a result, the reports become cumbersome and easily lose their effective- 
ness. In an effort to prevent this, reporting attempts to incorporate simple 
histograms, line charts, and numerical summaries, and only highlight out-of-control 
conditions, It is truly a challenge to present a brief report which contains the key 
elements and really captures management's attention. Actually, the purification of 
reporting to specifically meet your company's needs is a contiming problem. 


2. Statistical Studies on Product and Process Quality Problems 


jhe statistical techniques used in our production-engineering climate are the 
frequency distribution analysis, correlation analysis, significance tests, analysis 
of variance, and design of experiments. Some of our applications of each of these 
five techniques are described below; 
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TABLE 1 


Problem #1 - Close tolerance, leading defect, potential processing problem, or 
capability of process desired 


A- Customary Statistical Technique: Process Capability Study 


1.1 Select 100 to 200 consecutive pieces from an approved process 


1.2 Study the frequency distribution of readings for; 
X= 


a —xX m-/ 


where, 
arithmetic mean 
%,»%gsspecific observed values 
Mesample size 
2 standard deviation of sample 
2Standard deviation of population (es tima ted) 
6@<process capability 


1.3 Analyze frequency distribution for skewness and kurtosis (i ) 
2 (skewness) (Fe 


+ 


where, 


3 


1.4 Solve for the estimated pomlation fraction defective from sample results 
by performing the following: 


T 


Area from te 2, 
AacArea to 


where, 


A-A2=rea between specification limits (fraction effective) 
outside specification limits (fraction defective) 


B. 


afraction defective below lower specification limit 
1-A, afraction defective above upper specification limit 


Short-Cut Technique: Limited process Capability Study 


1.1 Select 15 consecutive pieces from an approved process 
1.2 Visually study frequency distribution for "normality" 


1.3 Determine range (2) of 15 pieces 


1.4 Estimate standard deviation of individual values (Y) from range (the range 
can be used for estimating the standard deviation only when the universe 
being sampled is near normal, and in the wide majority of industrial quality 
control situations, this is the case) 


T = (R)(o.-880) process Capebihty 


| — 
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TABLE 1 (cont.) 


1.5 Estimate the population fraction defective from sample results by referring I 
below to the circular nomograph for "grea Under Normal Curve." 


1.6 ‘he use of sample quasi-ranges in estimating the standard deviation can also 
be employed.to realize an increase in efficiency of the Y . For a sample 
size of 15 pieces, a 92% efficiency for estimating z can be realized by: 


(.15K6) (wo # 1.2011 


where, = range between largest and smallest mumber 
bat = range between second largest and second smallest number 


AREA UNDER NORMAL CURVE 


2 3 a $ € 2s 30 §9 Ble) 

/ 

STANDARD DEVIATION 


39.39% 


PERCENT ad AREA 


35% 
EXAMPLE / 46% OF TOTAL 
B=7,0:4 / AREA LIES IN 
BAND. 


NORMAL CURVE AREA NOMOGRAM 


€ 
8 
4 
/ 
B / 
BAND WIDTH 
/ 5 
/ 
/ 20 
/ 
/ 
/ 39 
/ 20 
/ en 
/ 
Oo 169 
/ 
/ 5% 
/ 
/ pal 
/ 
/ 
49.9% 
e / 
49.5% 
/ % 
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TABLE 1 Ccent.) 


Problem #2 - Significant defect, processing problem, important characteristic, 
or potential quality difficulty 


A- Customary Statistical Technique: Variable Control Charting 


2.2 Accumlate about 25 subgroups (sample sizes of 4,5 or 6 per subgroup) of 
measurements, where each subgroup is usually selected from the machine every 
few hours 


= 


2.2 a limits for Sean (X) and ranges (R) of subgroups 
R 


re, 
agrand average of subgroup averages 
Reaverage range of subgroup ranges 
@remmber of subgroups 
supper control limit for average chart 
slower control limit for average chart 
UCL,supper control limit for range chart 
Leti.aslower control limit for range chart 
Az. D,.Dg2 factors for average and range charts depending upon sample size of subgrouy 


as Ao = 0.73, Dd), s 2.28, D3 
N= 5, Ap = 0.58, D) = 2.11, Dz = 0 
n= 6, A, = 0.18, D, = 2,00, D, = 0 


2.3 Compare subgroup averages and ranges with control limits to determine con- 
stant cause system (valves within control limits) or assignable cause system 
(values outside control limits) 


2.4 Determine variability of individual measurements and estimate of process 


2 =esTimadle o recess 
’ 
aestimate of standard deviation of universe of individual measurements 
@average range of subgroup ranges __ 
d,2factor for estimating fron (R): 


as do = 

ne di factor assumes sam 
n 6, do = 2.53h f 5 


2.5 Compare the estimate of process (en to tolerance range, and take appropri- 
ate action. 


B. Short-Cut Technique: Limited Variable Control Charting 


2.1 Set up temporary control limits on the average (X) and range (R) charts for 
the given dimension without benefit of any prior information or without any 
preliminary process capability study. 


2.2 the tempor control limits are set up under the assumption that the six 
signa ) spread of the distribution obtained from the process coincides 
with the tolerance range of the drawing. Thus, if T is the drawing toler- 
ance range, then 


| 
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TABLE I (Cont.) 


= (42) (17) 


and for sample sizes of 5, where do = 2.326 
we obtain: 


= 0.3887 


2.3 Substituting this expression for — into the equation for control limits, 
we have; (For n = 5) 


x 
0.237 
x 
UCL, = 0.62T 
= 0 
where, 


Ty = assumed mean of drawing 
T = tolerance range 


2.4 Where the defect or quality problem has improved, variable control charts 
can be either removed, or contimed using a smaller subgroup sample size of 
ps 2 pieces. The control limits are now computed by calculating the 
from about 10 to 15 sample subgroups (n = 5) immediately preceding the 
adjustment in subgroup sample size, and solving for: 


v=%, 
Next calculate 
ucL_ =f, 4 av” 
Let 
UCL, 
ICL, =D, 
where, 
A = 2.12 
= 9 forns2 


Do = 3.69 
Yu = mean of drawing 


however, 

=T 
UCL_ X_7 0.23T 


Ae 
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TABLE I (Cont.) 


Problem #3 - Control of process and product mortality 
Customary Statistical Technique; attribute Control Chart 


3.1 To determine the presence of assignable causes of variation associated with 
fraction defective rates, consider the attribute control chart ("p*" Chart). 


3.2 Using the standard deviation of the fraction defective as Yee? 
compute control limits as follows: (assume p' = 


where, 


Pp = observed average fraction defective 

p’ = true process average fraction defective 

n = mumber inspected (daily, weekly, monthly, etc.) 
UCLp = upper control limit for fraction defective 


ULp = lower control limit for fraction defective 


Short-Cut Technique; Limited attribute Control charting 


3.1 Determine control limits for *p* char circular nomograph 
described below, which solves for 3\/ p(l-p 
250 300 


FRACTION DEFECTIVE 


300 
150 n 600 
SAMPLE SIZE 
: 700 
800 
100 900 
1000 
50 | 
~~ ~ = 3000 | 
025 | 
005 _ 02 03 048 40 20.50 
= 20 
10 
.005 
06 
05 
03 
O18 
025 
; 02 
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TABLE 1 (Con't.) 


Problem #l - Investigate the relationship between two variables, or the effect 
that the related variable x has upon the basic variable y. 


A- Customary Statistical Technique: Correlation Study 


4.1 Solve for the reoricd line, where 


.ope=b 
=x)* 


(m = pairs of observahens) 
REGRESSION LINE = 20 +bx% 


2 
CORRELATION COEFFICIENT =A = ae =x 


STRNDRARD ERROR estmate 


Be Short-Cut Technique; Limited Correlation Study 


4.1 Test whether two variables are correlated by checking the correlation 
between their ranks, using the Spearman rank - correlation test. 


4.2 Compute 


K (™ 41) = 


where, + <m fm 
m+i) (m— 
saver of pairs of observations 
squared differences in rank 
variable 


te 


.? a Cee but its mean and standard deviation are 0 
3 knowing K, you can refer to the table of the standard normal dis- 
of a value larger than any particular 
K. 


Problem #5 - Significance teste on data 
A- Customary Statistical Technique: Significance Testing 


5.1 Test to devermine if different samples belong to normal populations having 
the same mean and (Lo 


L.- (233s 


81° = within sample variance 

8,2 = variance based upon the derivation of all N observations about their 
mean & 
(The distributions of L, have been approximated and tables have been 
prepared). 

k mmeber of samples 


xt 
* 
INTERCEPT<«2= 4 
where, 
"4 
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TABLE 1 (Con't.) 


5.2 Test to determine if different samples belong to normal populations of the 
Same variance test) 


L, = (4335S )* 


where, 
s,? = within sample variance 
a,* = mean of the within-sample variances 


5.3 Test to determine the difference in variability in two independent samples 
selected from normal populations where the means are the same (F test) 


F = = 


where, 
S= standard deviaTion of Sample 
5.4 Test to determine if there is a difference between two sample means where 


the standard deviation cf the population is not known, tut assumed to be the 
same for both populations (t test) 


t = X, 
Yo 
A 


a\ 


where, 


estimate of standard deviation of the population 
= sample standard deviations 
b = sample means 


Short-Cut Technique: Limited Significance Testing 


5.1 A 95% confidence interval for the median * can be obtained from a sample (n) 
by determining the integer nearest to 


and then counting down this number from the sample largest reading and 
counting up this mumber from the sample smallest reading. It is necessary 
that the readings be ranked. 


* (If the distri mtion is sufficiently symmetrical, the median is a 
reasonable substitute for the mean). 


5.2 A test to compare the averages of two populations on the basis of two inde- 
pendent samples can be performed by the use of the Wilcoxon two-sample test. 
All observations of both samples are combined and ranked. ‘The approximate 
standard normal variable (K) is calculated by 


K = 2Rti-m 


500 


where, 
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TABLE 1 (Con't.) 


Sum of the ranks of either sample 
Sample sise from which R is taken 
mamber of observations in both samples 


22 >m(N-+1) 


+1 € 2R< m (N41) 
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(Having the value K, you refer to the tables of the standard normal distribu- 


tion to find the required probability. 
a value larger than any particular kK. 
are usually either upper-tail type or the two-tail type. 
to the significance attached to your K value, if K is 1.96 or greater, it 


Such tables show the probability of 
the probabilities in these tables 
AS a rough guide 


can be assumed that there is more than chance between or among the samples. 
This is considering a 95 percent probability that a normal distrituted var- 


iable is within / two standard deviations of the mean. 


95 percent for K = 1.96). 


That probability is 


5.3 A test to compare the averages of several populations on the basis of sev- 


eral independent samples can be had by employing the Kruskal-Wallis test. 


All observations of all samples are combined and ranked, The st 


normal variable is c ted from 
K 


Ry = 
> 
Ns 
ks 


(See 


NON+1) me ater 


Sum of ranks in i-th sample 
mumber of observations in i-th sample 
mumber of observations in all samples 
number of different samples 


the explanation above in 5.2? for handling K). 


= at 
Rs 
as 
a 
where, 

| 
“4 
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a. Frequency Distrimtion Analysis 


(1) Process capability studies 

(2) Dimensional changes of parts during various stages of processing 
(3) Ballistic results of ammnition 

(4) Performance results during environmental tests 


b. Correlation Analysis 


(1) Physical change in ammunition after heat-treatment 

(2) Dimensional changes in fire control and ammnition parts with 
the application of various processing techniques. 

(3) Relationship of ammnition accuracy between results at 300 yards 
compared to results at 600 yards. 

(4) Absenteeism of inspectors due to weather conditions. 


c. Significance Tests 


(1) Comparison of the difference in ammnition accuracy between 
defect and defect-free cartridges. 

(2) Comparison of the average and variability of product quality 
measurements from various sources ofapply. 

(3) Comparison between an expected test result and an observed test 
result for ammnition. 

(4) Comparison of the accuracy of decisions between types of 
inspectors. 

(5) Comparison of inspection results when using different test 
fixtures. 


d. Analysis of Variance 


(1) Study the effect that different ammunition and different guns ‘a 
have upon accuracy results. ‘ 
(2) Study the effect that working days have upon male and female 
inspectors. 
(3) Study the effect that different pull-test methods with bonded 
assemblies have upon pull test results, 
(4) Study the effect of machines and operators upon product quality. 


e. Design of Experiments o 


(1) Design of test to evaluate ammunition accuracy and gun effect at 
different ranges, considering order of firing ammnition in guns, 
method of cooling, and period of gun usage. 

(2) Design of test to evaluate a new process 

(3) Design of test to study fun. ‘ional results of new design under 
varying loads, temperatures, pressures, etc. 


The concepts and techniques associated with these five aforementioned statis- 
tical techniques can be had by making reference to the variety of excellent 
statistical textbooks which are available; some are listed in the reference of this 


paper. 
3. Review and Adjust Sampling Plans With Need 


Product variability, economics of inspection, actual dollar value of material, 
outgoing quality rates, changes in requirements, or modifications in processing, are 
sufficient to cause a re-evaluation of inspection plans for possible adjustment or 
replacement. The features of the "0-C" Curve, AQQ Curve, sample size, acceptance- 
rejection criteria, amount of Inspection Curve, and administration of the plan are 
studied once again, For example, consisicntly good, outgoing quality can trigger off 
a change from a single sampling plan on a class basis to a mlti-level contimos 
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Sampling plan on a characteristic basis, resulting in very noticeable savings in 
inspection man-hours. Or a single sampling plan possessing a discriminating "0-c" 
Curve will be replaced by a double sampling plan if we observe the presence of very 
good or very bad quality. ‘hen again, machine inspection with variable control 
charts might be reduced fram subgroup samples of 5 per hour to 5 per 2 hours or 1 
piece per hour, based upon the stability of the constant-cause system. Sampling 
plans by attribute can be considered to replace sampling plans by variables, reduc- 
ing sample sizes noticeably. Numerous other types of tailor-made sampling plans are 
modified or replaced frequently with the need, 


4. Variable Control Charts (¥ & R) on Leading or Potential pefects 


It is impossible to indiscriminately apply variable control charts in the 
production-engineering climate, The manpower needs are prohibitive. Therefore, 
attention is directed only towards quality problems, both before the fact and during 
the fact. Some typical problems for application of variable control charts are (a) 
leading defects, (b) close tolerances that can be the source of trouble (c) charac- 
teristics on components contributing to assembly difficulties, (d) past repetitive 
failures in meeting requirements on similar parts, (e) important physical or function- 
al tests, (f) machine groups that have been out-of-control on a fraction defective 
basis, and (g) new or modified machines. Normally,  & R charts are employed, with 
subgroup sample sizes up to ten (10) pieces; on limited occasions, X & Vcharts are 
used with subgroup sample sizes of twelve (12) to fifteen (15) pieces. 


Every control chart that is placed out in the shops is originally designed by 
quality control on a reproducible form (Figure 18) and then turned over to process 
inspection, along with reproduced copies, fhe time-saving features by reproducing 
copies has been found to be most significant. Control charts are maintained by 
process inspection, and monitored by quality control. When the need for such chart- 
ing is eliminated, the charts are immediately removed, 


5. audit product and Process Quality Results 


Complacency and rigidity of industrial operations is tantamount to industrial 
suicide: in today's market of complex eonipment. It is constantly necessary to con- 
tinue to audit performance to determine past, present, and future capabilities. 


From these determinations, various courses of action can be pursued, ranging 
from purely informational and advisory results to major changes in methodizing, 
mam facturing, or design. 


The extent of auditing includes; 


a = Weekly review and analysis of "Leading Defects" 

b - Monthly review and analysis of "Shop Process and Product Quality” 

c¢ - Monthly review and analysis of process averages and sampling effectiveness 

d - Monthly review of scrap dollar losses for shop and product 

e - Quarterly appraisal of inspection costs compared to scrap dollar losses 

f - Contimous monitoring of variable and attribute control charts maintained 
on quality problems or leading defects 

& - Periodic appraisal of adherence to quality assurance plans and procedures 

h - Analysis of outgoing product quality levels and problems 

i - appraisal of product and process quality from new processes 

j - Analysis of field complaints 


The results of this auditing serves as a basis for corrective action, adjust- 
ment in processing or inspection, and the realization of optimum performance, 


6. analysis of Quality Problems for Feedback to Design Engineers and Methods 
Engineers 


A good working relationship should exist among the design engineer, methods 
engineer, and Quality control engineer for the evaluation of product quality before 
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F. A. DIMENSIONAL QUALITY CONTROL 
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the start of regular production, during the stages of prototype mamfacture and in the 
course of regular mam facture, 


Our quality control engineer contributes in the feedback cycle to design and 
methods engineers by working on the -- 


- realism of specifications, 

practicality of tolerances, 

test sample sizes, 

- evaluation of inspection and test results, 

compatibility of tolerances with machining capabilities, 

- quality control planning, 

- design of industrial experiments, 

- developing of a quality index or a composite picture of the final product 
quality, and 

i - application of advanced gaging. 


In actuality, the contributions made by the quality control engineer are both 
engineering and statistical. The closer the quality control engineer remains to the 
new or modified design, the more effective is the prevention of quality problems, 
Concentrated effort devoted to the details in the early interval of the product growth 
prevents the utilization of mch trouble shooting during mamfacture. 


7. Analysis of Costs of Achieving and Maintaining Product and Process Quality 


The acid test of effectiveness most frequently used by management is the cost of 
operating the activity and the resultant savings. The economics of quality in the 
Operations Division is primarily considered from the standpoint of dollar value of 
scrap compared to the dollar value of production dollars expended. Comparisons are 
maintained and studied monthly for each of the major mamfacturing shops and for each 
major type of production item. Actually, the comparison is expressed in terms of a 
percentage index (I); 


= 


where 


I = % (index) 
D = accumlative dollar value of scrap material 
P = accumlative production dollars expended 


Each month, the complete dollar picture is presented to management, with appropriate 
recommendations. From this economic appraisal, various courses of action are pursued. 
This might include adjustment in inspection for more effective control measures at the 
operation causing the significant defect, closer supervision devoted towards certain 
products or operations, re-evaluation of the process by methods engineering, differ- 
ent tooling needs, chenges in material, etc. in Figure 19 can be seen the graphical 
portion of the "Monthly Scrap Dollar Index" report, which also includes man-hours 
lost in scrap compared to man-hours expended, The economics of quality losses prove 
to be a good, tangible score readily understood by all levels of management and 
supervision, 


Conclusion 


There is no one best way to controlling quality in the production-engineering 
type of industry. On the other hand, the science and management of quality control 
has advanced sufficiently to offer mamerous ideas and techniques for coping with par- 
ticular situations. [In this paper, an attempt has been made to describe a particular 
approach to the control of quality in an industrial climate where the planning, con- 
trol and mamfacturing are faced with a wide variety of low density items which range 
throughout all degrees of complexities, Practical quality control applications, 
together with theoretical techniques, have been tailor-made to meet the need. 
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This discussion has been presented in the sincere belief that regardless of the 
system you employ in controlling and assuring quality, there mst be integrated and 
systematic thinking. The increasing demands for precision in military and consumer 
goods necessitates that qality control people effectively design their operations. 
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A QUANTITATIVE APPROACH TC CLASSIFICATION OF CHARACTERISTICS 


Benjamin W. Marguglio*, Sr. Quality Engineer, Quality and 
Reliability Department 
The Martin Company, Baltimore, Md. 


Martin W. Sullivan, Section Head - Planning and Analysis, 
Quality Control Department 
Raytheon Company, Andover, Mass. 


INTRODUCTION 


"Quality" refers to the ability of the product to meet given specifications 
and comply with design drawings. “Quality of a product" should not be confused 
with "a quality product," the latter having a generally accepted connotation of 
a luxury item. 


The quality of a product can be measured only in relation to its quality 
characteristics. By classifying the importance of each characteristic and in- 
specting to an adequate corresponding protection level, three substantial ad- 
vantages accrue: 


1. Quality is improved by affording each characteristic the proper 
inspection emphasis; 


2. Inspection costs in relation to improved quality are reduced by 
de-emphasizing those characteristics of lesser importance; 

3. Design is retained by the design agency. In the absence of class- 
ification of characteristics, each inspector is prone to accept 
or reject based on his own evaluation of the importance of the 
non-conforming characteristic, thereby altering the design 
criteria. 


In most applications to costly, complex end products, characteristics are 
usually classified within the following definitive framework: 


1. Critical - Very Serious - a departure from requirements of such a 
characteristic, if undetected by inspection: 


a. Could cause an operating failure of the system; 
b. Could result in reduced operating life of the system; 
¢. Could cause personal injury or property damage. 


2. Major - Serious - a departure from requirements of such a char- 
acteristic, if undetected by inspection: 


a. Could cause a system failure of a less serious nature (e.g., 
adjustment failure, operation below standard); 


b. Could cause a failure of the unit (not of the system) or re- 
duce the usability of the unit; 


c. Could affect interchangeability and assembly operations; 
d. Could necessitate increased maintenance. 

3. Minor - Not Serious - a departure from standards of such a char- 
acteristic, if undetected by inspection, although undesirable, 


would not appreciably interfere with the function or usability of 
the unit. 


* Written at Raytheon, Andover, Mass. 


ASQC LCS Code 213:20:00 


| 


510 1961 ASQC CONVENTION TRANSACTIONS 


THE QUESTION OF DEGREE 


The above definitions are qualitative in nature. Conspicuously lacking is 


a quantitative guide to aid the classifier in his judgement. 


from the cognizant engineer might be paraphrased as foll 


acteristics become critical if their tolerance is sufficiently exceeded?" 


An obvious question 


ows: "Do not many char- 


Surely 


it is true that an amplifier gain specified as at least 1000 could become criti- 
cal if it were only 500, but the probability of such an occurrence not being de- 


tected by inspection is very slim under usual sampling protection levels. 


The problem encountered by the engineer is therefore one of degree. 


Nat- 


urally, if the engineer decides that a characteristic would be critical when its 


tolerance is exceeded. regardless of the degree, that characteristic should be 


classified as critical. This situation does not present a problem. 
the engineer's dilemma arises when a characteristic can be either major or 


However, 


eritical - major when its tolerance is exceeded by less than a specific amount, 


eritical when its tolerance is exceeded by more than that specific amount. 


Prone to think in terms of safety factors, the engineer would probably 


classify all such characteristics as critical, thus necessitating undue inspec- 


tion emphasis and cost. 


ANALYTICAL DECISION BETWEEN CRITICAL AND MAJOR CLASSIFICATION 


An example would best illustrate the suggested quantitative approach to the. 


above problem. Assume the following: 


1. Incoming sampling inspection in accordance with MIL-STD-105B, 


Level II, Normal Inspection, .65% A.Q.L. 


2. 100 per cent inspection of rejected lots. 


3. Am average incoming lot size of 100 units (necessitating a sample 


size of 25 with a rejection number of 1) with the characteristic 


in question classified as major. 


4, A drawing requirement for that characteristic of .750" with a 


tolerance of + .005" (.745" to .755"). 


5. A normally distributed incoming lot of 100 units with this char- 


acteristic averaging .750" and varying from .7395" to .7605" - an 


excess of tolerance of approximately 110 per cent determined as 


follows: 


[(-7605 - .755) + .005] [100%] = 110% 


Under this set of circumstances the lot would be 9.56 per cent defective 


as diagrammed in Figure l. 


Theoretical 
specification 
distribution 


4.78% def 


0.7395 0.745 0.750 0.755 


Fig. I--Percentage Out of Tolerance 


Distribution of 
incoming lot 


4.78% def 


0. 7605 
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Standard deviation of incoming lot - estimated from range using 7 
sigma as representative of the distribution's total spread:* 


se = .0210 +7 
= .003 
Per cent of lot within tolerance (Q,): 
t = (.750 = .745) + .003 = +1.667 = 45.22% 
t = (.750 - .755) + .003 = -1.667 = 45.22% 


Q= 45.22% + 45.22% = 90.44% 
Per cent of lot out of tolerance (P,)s 
= 100.00% - 90.44% = 9.56%** 


The above assumed sampling plan would, for a 9.56 per cent defective lot, 
yield a probability of rejection of approximately .952 as computed from the 
hypergeometric distribution. 


Lastly, assume this .952 probability of rejection to be an adequate level 
of protection against accepting a critical departure from standards. 


This simple analysis affords the engineer a quantitative tool to aid him 
in his decision between a critical and major characteristic. Applicable only 
to the above stated sampling plan, if the engineer decides that a characteristic 
would be critical only when its tolerance is exceeded by 110 per cent or more, 
that characteristic should be classified as major. 


Thus, for this sampling plan, if the characteristic were to exceed the 
drawing tolerance by anything greater than 110 per cent, the lot would be 
greater than 9.56 per cent defective and would, with very high probability, be 
rejected. All critical departures would be screened in 100 per cent inspection. 
If the characteristic did not exceed the drawing tolerance by 110 per cent or 
more, it would not be critical by engineering decision and would be subject to 
the protection level for major characteristics provided by the sampling plan. 


It must be stressed that the use of 110 per cent in excess of the tolerance 
should not be considered as gospel. Rather it signifies a protection level 
thought to be adequate. The protection level offered by each percentage excess 
of the tolerance varies with different sampling plans as well as for different 
lot sizes within the same sampling plan. 


In Figures II, III and IV, Classification of Characteristics Risk Curves 
are given for .40%, .65% and 1.0% A.Q.L.'s respectively. Again, an example best 
illustrates the use of these curves. Assume a .65% A.Q.L. sampling plan in 
existence for major characteristics with a minimum lot size of 100 for units in 
question. It is desired to determine that percentage excess of tolerance that 
will yield a .99 probability of rejecting the lot (.0l1 probability of accept- 
ance) that contains one or more critical deviations. A greater risk cannot be 
tolerated. From Figure III, curve F, 140 per cent is found to meet the criteria. 
Therefore, in classifying characteristics, the engineer should: 


1. Classify as critical those characteristics which will become 
critical when their tolerances are exceeded by less than 140 per 
cent. 


° The use of 7 rather than 6 sigma to estimate the standard deviation results 
in a smaller egtimate of the lot per cent defective and, therefore, a smaller 
probability of rejection. This is a conservative approach in that the prob- 
ability of rejection is more likely to be underestimated than overestimated. 


** The estimated per cent defective of the incoming lot is based on an equal 
proportion beyond each tolerance limit. If the distribution exhibited a 
mean shift, the per cent defective would be greater and hence the probability 
of rejection would be greater. 
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2. Classify as major those characteristics which will become critical 
only when their tolerances are exceeded by 140 per cent or more. 


ANALYTICAL DECISION BETWEEN MAJOR AND MINOR CLASSIFICATION 


When a characteristic can be either minor or major - minor when its toler- 
ance is exceeded by less than a specific amount, major when its tolerance is 
exceeded by more than that specific amount - a similar dilemma confronts the 
engineer. Using the applicable sampling plan for minor characteristics, the 
approach, as before, is merely to determine a percentage excess of tolerance 
that will yield a per cent defective corresponding to the desired probability 
of rejecting a lot that contains one or more major deviations. Having deter- 


- mined this corresponding percentage excess of tolerance, the engineer would: 
1. Classify as major those characteristics which will become major 
1 : when their tolerances are exceeded by less than that determined 
percentage. 


2. Classify as minor those characteristics which will become major 
only when their tolerances are exceeded by that determined per- 
centage or more. 


In Figure V, Classification of Characteristics Risk Curves are given for a 
4.0% A.Q.L. which is very common sampling level for minor characteristics. 


CONCLUSION 


What has been done in this paper can be summarized very simply. Per cent 
defective has been converted to percentage excess of tolerance, thereby convert- 
ing operating characteristics curves to classification of characteristics risk 
curves. 


To the authors' knowledge little, if anything, has ever been published on 
a statistical approach to classification of characteristics. The need exists 
for such an approach especially if industry is to realistically meet the in- 
creasing military requirements for classification of characteristics. 


Granted, there are many uncontrollable factors such as differing engineer- 
+h ing opinions, differing lot distributions and varying costs to mention the more 
: important ones. Nevertheless, this technique provides estimates that are ade- 

quate for their purposes. 


Reference: 


Brown, Dewitt and Sullivan, Martin; Design Engineering Familiarization 


Presentation E.0.D. Quality Control Plan; General Electric Corp.; 
Cincinnati, Ohio; November, 1955; unpublished. 
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A-N = 5000, n = 225, 
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*This plan corresponds 
to Inspection Level 0, 
Normal Inspection as 
given in MIL-STD 105B 


Probability of Acceptance 
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Percentage Excess of Tolerance 
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Fig. Il--Classification of Characteristics Risk Curves, 0.40% AQL* 
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Fig. IlI--Classification of Characteristics Risk Curves, 0.65% AQL* 
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Fig. V--Classification of Characteristics Risk Curves, 4.0% AQL* 
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SUBJECTIVE PRODUCT EVALUATION METHODS 


E. T. Zlobik 
General Foods Research Center, Tarrytown, New York 


Throughout the various phases of developing and processing food products, there 
is as much need for frequent evaluations of taste as there is for physical and chemi- 
cal measurements. Even in the early stages of development, the food researcher must 
answer or obtain answers to questions such as “What happens to flavor with tempera- 
ture increases?", or “What are the differences among a group of experimental form- 
lations?", or "Which formula does the consumer prefer?" These and many other related 
questions ere asked in each succeeding step of producing foods. They will fall into 
two general categories, one pertaining to difference measurements and the other to 
preference measurements, and will apply to new product development, product improve- 
ment, cost reduction problems, and quality control problems. 


Let us first examine difference measurements. 


Among many techniques available for determining whether two or more samples 
differ from each other in one or more ways, there is one known as multiple choice 
tests. These tests consist of asking judges to identify or group samples into classes, 
e.g., judges may be told that among six samples, three are alike and three are differ- 
ent. They are then asked to separate the samples into two groups of three. A vari- 
ation of this would be presenting five samples to judges, instructing them that four 
are alike and one is different and asking them to identify the different sample. In 
each case, the probability of grouping or identifying correctly by pure guess is 
readily calculated (1/10 for the first example, 1/5 for the second). If the differ- 
ence among samples is real, or if the judges have an ability beyond a mere guess, 
then the proportion of correct answers will be significantly greater than the chance 
probability. A notable example of this type of test is the test variously know as 
the triangle, odd sample, triad, delta, or trio test. It consists of presenting a set 
of three coded samples to a panel of judges in which two of the samples are duplicates 
and one is different. Judges are informed that two are identical and one is different, 
and they are asked to identify the different one. If the number of correct identifi- 
cations is significantly greater than chance, it is concluded that the two samples are 
different. Known sources of bias, e.g., code and cup position, are reduced, in the 
first case, by random selection of code letters and, in the second case, by presenting 
cups in all possible positions an equal number of times. 


This type of test has the advantages that it 


1. is easy to administer; 

2. requires a minimum of training of judges; 

3. causes minimum concern regarding taste fatigue or adaptation; 
4. lends itself to simple, clear-cut statistical analysis. 


Some of the shortcomings of the triangle test are: 


1. The chance expectancy level is relatively high, i.e., one of three successes 
can be due to chance. 

2. The relationship of a group of samples to some standard in terms of magni- 
tude of difference cannot be established. 

3. No information can be obtained on how samples differ, e.g., "Is coffee A 
more sour than R?" or "Does orange juice A have a higher flavor intensity 
than B?’ 


A very efficient way in which to conduct and analyze triangle tests is by the 
use of a sequential sampling plan. These plans are efficient from both the opera- 
tional and statistical points of view. we are presently using a plan wherein judg- ‘ 
ments are obtained in blocks of twelve and are run to a maximum of sixty judgments. A - 
block consisting of twelve judgments allows for complete balancing of all six possible 
positions in which samples may appear. 


The results of each block of judgments are examined for significance and one of 
the following conclusions is reached: 
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1. The samples do not differ; 
2. The samples do differ; 
3. Additional information is needed. 


In the first two cases, testing is stopped, while in the third, another block of 
twelve judgments must be obtained. 


Two testing situations, each involving two types of error, are set up in this 
plan, and a reasonable number of judgments is obtained for meaningful results. Type 
I (or a) error is defined as the risk of saying there is a difference where none ex- 
ists, while type II (or 8) is the risk of failing to find a difference where one does 
exist. 


In the first situation, which is applicable to quality control problems, we are 
interested in a small type I error and can afford to have a somewhat larger type II 
error. The second situation is applicable to product improvement or development prob- 
lems. Here we are usually interested in a small type II error and are willing to 
have a slightly larger type I error. In other words, we want to take a small risk of 
failing to find differences where they exist. If wrong answers should be.obtained by 
finding a difference where none exists, we merely put ourselves into the relatively 
harmless position of running additional, yet unnecessary, tests. 


Another test of the multiple choice class is the duo-trio. 


Briefly, the duo-trio consists of giving a labelled control to the judge and then 
presenting two "samples" under code, one of which is the control and the other the 
test sample. The judge is asked to indicate which sample matches the control. The 
test sample and the coded control are given in random order successively, and tasting 
is timed. This test is less efficient statistically than a triangle test because the ; 
chance expectancy is greater, i.e., 1/2 instead of 1/3. However, for some types of ‘ 
products it can be more effective than a triangle test. For example, in whiskey 
testing, where carry-over of flavor and taste fatigue are apt to be a problem, con- 
trol of the time interval will help to reduce these biases. Also, the test is a more 
simple one from a psychological point of view since the judge is presented with only 
two unknowns rather than three as in the triangle test. 


Another category of difference testing is the ranking test. This type of test 
works well where unidimensional differences are being tested. Suppose there are five 
sugar solutions of concentrations X,, X,, a » X,, and X,- These concentrations are 


set up to form a geometric progression. The reason for this is that taste supposedly 
is linearly related to concentration on a percentage scale. Five coded samples are 

presented to a panel of judges, each of whom ranks them in what he thinks is the in- 
creasing order of concentration. Alternatively, if the concentration x > X, » X, and 


X, are known, a fifth sample of an intensity believed to be between X and X, is pre- 
sented, and the judges are asked to rank this unknown among the known concentrations. 


The greatest disadvantage of this test is its limited application, particularly 
in the food field. Most of the time situations are not such that single dimensional 
differences occur. In a complex such as vanilla instant pudding, if formulation 
changes are being made where sugar level is the variable, characteristics other than 
sweetness may imfluence the taster. There may be texture differences, or the aroma- 
tics may appear less or more intense, in addition to differences in the basic taste 
of sweetness. 


Another test that is very useful in measuring unidimensional differences among 
samples is the paired comparison test. This test consists of presenting a series of 
paired coded samples to the jydge and asking him to indicate the relative concentra- 
tions of the variable in each pair. The order of presentation of samples is random. 
An application of this technique can be found in a paper by J.E. Sinsheimer (1). 


The advantage of this test is its psychological simplicity. The judge is asked 
to make a decision on only two samples and is given specific information as to how 
the samples differ. This, like the rank test, has the disadvantage of limited use, 
particularly in food testing, where the rule is testing samples of a complex rather 
than a simple nature. 


The multiple choice tests, ranking tests, and paired comparison tests give in- 
formation on whether samples differ from each other without regard to the magnitude 
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of difference. Sometimes it is important to obtain this type of information. For 
example, samples differing in intensity of flavor level may be submitted for determi- 
ning the degree of difference among them. The test know as “difference-from-control" 
can be used in this case. 


The difference-from-control test consists in giving the judge a labelled control 
and coded test samples and asking him to indicate the degree of difference from the 
labelled control by use of a rating scale ranging from “identical with control" to 
“very different from control". The number of steps on the scale is not standard. 
However, 5-10 gradations are most frequently used. Less than five steps tend to be 
too coarse to show relative differences, particularly where small differences exist 
among samples, and more than 10 steps tend to confuse the judges. 


It is useful to include a blind control in this test since it acts as an internal 
check of judge consistency. It also gives practical meaning to the mean scores 
assigned to the test samples. For example, if the blind control should receive a mean 
score of 2.0 on a 10 point scale, only samples whose ratings differ from 2.0 are con- 
sidered as distinguishable from the control. 


Generally, a total of five samples is a good number to be presented to a judge. 
More than this would dilute the results because of physiological or psychological 
fatigue. 


There are certain difficulties, however, associated with using rating scales. 
It is difficult to obtain a reliable mean score since the judges will not necessarily 
interpret the scale in the same fashion. That is, the difference between two samples 
may be termed “slight" by one judge and "noderate® or “large” by another. It has also 
been my observation that individuals tend to fall into a pattern of using a certain 
part of the scale during several weeks of testing different types of products or the 
same type of product, regardless of the range of differences presented to them. 
This phenomenom may heve something to do with the personality of the judge. Shy or 
conservative people may hesitate to rate samples “very different", while a strong- 
minded or dogmatic type may always use this category. 


There is another difficulty associated with this test, namely, that the rating 
scale is a unidimensional measurement that does not indicate direction. Let us 
suppose that three experimental coffees differ from the control as follows: 


1. Sample A is moderately different from control on the basis of less bitter- 
ness. 

2. Sample B is moderately different from control on the basis of more bitter- 
ness. 

3. Sample C is moderately different from control on the basis of less sourness. 


Assume that the panel is adequately scoring the sample in terms of magnitude of 
overall difference, i.e., A, B, and C are moderately different from the control. We 
have no way of differentiating among the three samples because they all received a 
mean score reflecting moderate difference. However, they are, in fact, different in 
different ways. Therefore, a conclusion that all three are moderately different from 
the control might lead one to erroneously believe that they are probably similar to 
each other. 


There are also tests for screening and rating judges. 


For general information on judge taste acuity, there is no specific type of test 
that will do the job better than any other. All of the tests discussed here for test- 
ing samples can be used for testing judges. As a matter of fact the testing of judges 
and samples can be done concurrently. Ome example where this has worked well in 
practice is the triangle test (2). The analysis of judge ability is based on the pro- 
portion of correct identification in the test. The judge who misses on an easy test 
is penalized more than one who misses on a more difficult test. Conversely, the 
judge who identifies correctly on an easy test receives less of a score than one who 
identifies correctly on a more difficult test. The scores are standardized to take 
into account the number of tests in which a judge participates. Thus, a judge can be 
evaluated on a relative basis regardless of the type or number of tests in which he 
participates. 


A similar situation can be worked out for difference-from-control tests, ranking 
tests, paired comparison tests, and others mentioned above. 
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In specific cases, where it is of interest to the taste ability of a panel for 
a particular product or situation, it is suggested that that test method be selected 
which best suits the situation, it is suggested. 


For example, if the primary interest is in testing simple properties, such as 
bitterness or sweetness, then the rank or paired comparison test would be preferable 
to the triangle. However, if the product happens to be coffee or same other complex, 
the triangle would probably be more useful because it would be difficult, if not im- 
possible, to instruct judges on the specific difference which they are expected to 
evaluate. 


A classical test method for evaluating judges is the threshold test. This is 
usually not used ‘in product testing. It was designed primarily to obtain information 
on physiological sensitivity to a given substance. Generally, such tests are run on 
the four basic tastes, namely, salt, sweet, sour, and bitter, to determine at what 
level an individual or a group is able to detect and identify the stimuli. This test 
has limited practical application, particularly in food testing, since most products 
are made up of components which are in the super threshold range and there is no 
clear-cut relationship on judges’ ability in the threshold vs. the super threshold 
range. It has been my experience that there are judges who differ appreciably in 
their thresholds to "bitter" but are about the same when evaluating coffee containing 
a fair amount of bitterness. 


All of the tests discussed thus far are useful in determining whether or not two 
or more samples differ on an overall basis or in some specific, known characteristic. 
However, they do not answer how samples differ in many unspecified characteristics. 

For example, processing studies comprising different treatments (temperature , pressure, 
etc.) may produce multi-dimensional flavor differences in samples which are not pre- 
dictable by the experimenter. A technique available for obtaining this type of in- 
formation is called "flavor profiling". 


The flavor profile is a semi-quantitative method wherein differences between 
samples can be judged on the basis of intensity of individual character notes, order 
of appearance of the notes, aftertastes, mouthfeelings, blend, and body. Samples may 
be found different in all or some of these factors. This method is as follows: 


The panel evaluates each sample in terms of the above mentioned characteristics. 
Each judge gives his results verbally to a panel moderator. A group discussion of 
the results then takes place. If any major disagreements are found, the panel meets 
at another session to resolve the differences. 


The following is an example of two coffees which are shown to be different by 
profiling: 


Coffee A Coffee B 
Aroma Aroma 
Character Notes Intensities Character Notes Intensities 
burnt (bean + cereal) 2 charred cereal 1-2 
coffee fragrance 1-2 pyridine 1 
burnt (caramel and vanillin) 1-2 heavy sweet 1 
sour 1 
Flavor-by-mouth Flavor-by-mouth 

Character Notes Intensities Character Notes Intensities 
sour 1.2 sour (unclean 2-3 
burnt (cereal) 2 charred cereal 1-2 
bitter 1-2 bitter 2 
coffee fragrance (old boiled) 1 astringent 2 
burnt caramel 1 

Intensity Scale 

3-strong 

2-moderate 

l-slight 

X-barely perceptible 

O-none 


It is fairly obvious that Coffee A differs from B in character notes, quality of 
these notes, and intensities. Coffee A has a coffee fragrance note in both aroma and 
flavor which is not found in B. Coffee B has a pyridine note in aroma not found in 
Coffee A. Coffee B is described as having a charred cereal note, whereas A has a 
burnt bean + cereal complex. Coffee B is astringent, whereas A is not. Coffee B is 
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more bitter than A. 


Panels applying this technique must be thoroughly trained in the broad aspects of 
flavor perception. For training, a group of five to seven people who have normal 
abilities to taste and smell and high interest undergo a short period of indoctrina- 
tion in the fundamentals of flavor perception. They learn something of the physiology 
of taste and smell and of the relationship of chemical and physical factors to flavor, 
a relationship which unfortunately is not clear-cut. Through group discussions end 
by use of physical reference standards, they learn something of the techniques for 
evaluating different types of products and the importance of developing a common and 
meaningful vocabulary for describing what they perceive. After this initial indoctri- 
nation, the group spends some time applying the basic principles to carefully selected 
practice samples. 


In speaking about the qualifications of such a pane], I mentioned normal ability 
to taste and smell and high interest. Most people have normal physiological ability. 
However, interest is something else. The necessity for interest cannot be over-empha- 
sized. In my experience, it is a major factor in the success or failure of this type 
of panel operation and takes equal place with panel training. 


Some of the advantages of the flavor profile method are: 


1. Multi-dimensional differences in samples can be described in terms of quali- 
tative and quantitative scores. 

2. The results can supply direction to the experimenter in improving his prod- 
uct, e.g., if roasting time or temperature studies are being conducted on 
coffee, the panel can select those samples which have the roast character 
desired by describing the individual characteristics in terms of some stan- 
dard. In accelerated storage studies, it is often difficult to determine 
gradual staling or deterioration in a product, unless these results are 
given in terms of specific descriptive differences rather than overall diff- 
erences. 


Some of the disadvantages to this method are: 


1. Initially it is expensive because of the amount of training time involved be- 
fore the panel can be used in an actual work situation. 

2. The results do not lend themselves well to statistical analysis or design. 

3. The personalities of panel members or panel moderator might have an influence 
on the final results of how samples differ. 


We have been discussing test methods which are very effective means for evalua- 
ting differences (both overall and specific) among samples. Up to this point, how- 
ever, we have no information regarding the public's reaction to our new or improved 
products. Since laboratory personnel are examining products in a detached and scien- 
tific manner and since they are more or less trained to evaluate food products, they 
are not likely to reflect the preferences of the consuming public. Therefore, prefe- 
rance tests are conducted in which the consumers are the judges. The type of test 
most frequently run to obtain preference opinions is a home-use test. Basically, this 
test usually involves sending samples to a fairly large number of families (100 or 
more) and asking each family member to taste the samples either side-by-side or indi- 
vidually at different meals and to express a preference or no preference. There are 
many variations in this type of testing. Samples can be placed by mail or personally; 
various types of questionnaires can be designed for the particular information re- 
quired; opinions on sample preparation may be obtained from the housewife; child- 
ren's preferences can be examined separately from adults; and testing can be re- 
peated over an extended period to determine consistency in consumer opinions or wear- 
ability of product. The advantages of testing products in the home are quite obvious. 
The products are being evaluated under fairly realistic conditions, the housewife has 
an opportunity to evaluate the product from the preparation and taste standpoints, 
and opinions can be gathered from all family members. 


Most companies use the services of the various opinion research, market research, 
or advertising agencies for obtaining this type of consumer judgemnt. 


Sometimes it is desirable to obtain consumer opinions on samples before they 
have reached the point of completion in terms of recipe tolerance. A home-use test 
would not be feasible under the circumstances since one can hardly present samples to 
the consumer which are prepared by pipetting a few milliliter of a flavor into proc- 
uct base. There is a method available for obtaining opinions from the consumer on in- 
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complete products. Booths can be set up in an area with fairly heavy traffic, such 
as department stores or supermarkets, where samples can be prepared by interviewers 
and handed out to shoppers. Most often, two coded samples are presented to the par- 
ticipant and he is asked whether he has a preference or no preference. Sometimes 
participants are asked to comment on why they preferred one sample to another. 


It is realized that the individuals who are testing are not necessarily a good 
sample of the population about which generalizations are to be made. Since it is 
only preliminary testing, this lack of representation does not entail too great a 
risk. 


Because of the increasing interest on the part of food companies toward marketing 
brand-new items which have no counterpart in a homemade recipe, there has arisen the 
need for more detailed information than is normally obtained in a home-use test or an 
on-the-spot preference test in a department store. A considerable amount of time has 
been devoted to interviewing homemakers in detail about their attitude toward the 
concept of a product as well as how much they liked the specific samples presented to 
them. 


One such way of conducting this type of test is called the round-table method. 
A group of five to ten homemakers are usually handled at a session. They are pre- 
sented with one, two, or more samples of a product, asked to examine them on the basis 
of flavor, texture, visual characteristics, etc., and to fill out a ballot expressing 
their opinions. A moderator then asks each participant to express her opinions of 
the samples she evaluated and to make any other comments that she might like to ex- 
press. Often, these round-table tests are conducted subsequent to a home-use test, 
i.e., the round-table participants were also members of the home-use test panel. At 
times, much useful information can be gathered from this interview method, which 
would not show up in answers to questionnaires sent through the mail. A very effi- 
cient way of recording the discussions and conclusions of a round-table group is by 
means of a tape recorder. The moderator can conduct the session comfortably and 
efficiently without being concerned that important points have been overlooked in her 
own notes or those of a stenographer. 


Despite the numerous and diversified schemes available for measuring difference 
and preference, there is still much to be done in the field of subjective evaluation 
methods both from the point of view of improving current methods and developing new 
methods. One area of investigation that might prove fruitful is learning more about 
human response; this will subsequently lead to better interpretation of test results. 
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PUTTING THE SENSES TO WORK IN QUALITY CONTROL 


Peter van S, Rice 
Treasurer & General Manager, A. H. Rice Company 


Measurement is one of the foundations on which modern quality control rests, Conditions 
for the optimum quality control situation can exist only when a product characteristic can be de- 
fined in a manner which precludes any possibility of misinterpretation. Where this is attained, 
both buyer and seller have complete assurance that their respective interests can be protected 
beyond any shadow of a doubt, Unfortunately, however, there are too many cases where this 
goal has not yet been attained. In some, the test method is imperfect, in others the quality 
characteristic involved is not capable of being reduced to measurable or numerical values, It 
is with the problems associated with the latter area that this paper will deal. 


SPECIFYING QUALITY AND SETTING THE STANDARDS 


Before standards can be set up for any product characteristic, there must be agreement 
between buyer and seller as to exactly what is desired or needed, In other words, the goal must 
be defined. This, of course, is true for all characteristics, and not merely the non- measurable 
class being considered here, Standards for measurable quality characteristics are capable of 
being specified and defined in written terms - both the test method and results which are to be 
obtained to meet quality requirements, Unfortunately this is impossible in the case of non- 
measurable characteristics. The only successful method by which adequate standards may be 
specified for this class is to set up a standard sample to which questionable products may be 
compared, Unless this is done, the standard can exist only in people's memory or in their 
interpretation of written description. Interpretation obviously is subject to many shades accord- 
«ing to the conscious or subconscious needs and caprices of the individual concerned, 


But it is also essential that the standard sample be supplemented with a written descrip- 
tion standard. This description, in effect, defines the characteristic, while the sample is the 
measurement of that characteristic. Needless to say, the conditions under which the sample is 
to be compared - i.e. , the test method, must also be defined, 


A further step which is all too seldom taken, but one which is obviously desirable, is to 
set up tolerance standards. One of these standards will represent the quality level that is de- 
sired - the optimum; while the other represents the level below which the product is not accep- 
table, In some cases this range of tolerance may be quite narrow. In others it may be broad 
enough to make it worthwhile to justify a third sample which would represent, say, the average 
quality desired in a. iat or shipment with the others being the maximum - minimum levels for 
individual units, Additional samples can also be added to the standards to represent other 
specific levels of quality. 


The purpose of such tolerance standards is, of course, to help minimize the problem of 
decision in cases of marginal product quality. Daily cases will be found in every plant where a 
product or lot is very close to the standard - so close as to give the inspector fits as to whether 
to accept it or reject it. These tolerance standards do not eliminate this border-line problems, 
but they do help the inspector decide how far he can go. 


Only a short step beyond the setting of tolerance standards is the establishment of rating 
procedures. Such procedures require a series of from four to five standards representing 
quality classifications ranging ftom very poor to excellent, The maximum number is set ai five 
because the human being cannot normally classify sensory reactions consistently into more than 
five grades, 


Rating methods will not be discussed here except to note that they provide a means for 
graduating quality beyond a "pass-or-reject” classification. They are designed to grade product 
quality along a continuous scale so as to indicate trends of quality above or below the reject 
point, Such procedures are essential if non-measurable quality conditions are to be included 
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in any well developed quality index system, 


Before leaving the matter of standard samples, a brief discussion of their selection may 
be helpful. If only one standard is to be used, the procedure for selection is simple; the parties 
negotiate between themselves in determining what shall be considered acceptable from a group 
of samples representing various quality levels, 


Where two or more tolerance standards are to be selected, a different procedure must be 
followed, For the problem here is not merely one of drawing a line between good and bad, but 
rather a problem of selecting samples to represent each of the quality levels agreed upon. The 
samples are, of course, selected by the interested parties - by Production and Sales, by Engin- 
eering and the customer, etc, Usually the Quality Control Department will participate as tech- 
nical advisor. A proven way of selecting such standards is to arrange in random order a group 
of samples which represent a range of quality of the characteristic in question. Then each party 
chooses samples which he feels possess the correct quality for each agreed upon point in the 
tolerance range, Needless to say, it will be rare that two or more parties will agree at this 
point, Therefore, the next step is to arrange the samples again, this time in ascending order 
of quality. The individuals concerned will then be able to visualize the quality range of the 
characteristic, and will thus be in a position to negotiate in the final selection of mutually 
agreeable standards, The method for the selection of samples for use in an attributes rating 
program will follow the same procedure outlined above. 


These standards, once approved, should be accurately duplicated and the resulting samples 
which could be called "reference standards" forwarded to all parties or departments that will be 
concerned with quality assurance, For example reference samples in the manufacturing and 
inspection areas of a plant are of inestimable value in providing answers, to say nothing of 
settling arguments, on quality standards as soon as they occur. Needless to say extreme care 
must be taken to protect the reference samples from deterioration through exposure and use, 

A formal system should be developed for periodically checking them against the approved 
master sample, which, of course, is kept under lock and key! 


ADMINISTERING THE CONTROL PROGRAM 


Once meaningful and realistic standards have been established for non-numerical quality 
characteristics, the administrative problems of assuring conformance to specifications are 
those typical of Attributes Inspection as distinct from the administration of Variables Inspection. 
Inspection by Attributes, of course, is defined as inspection wherein a unit of product is classi- 
fied as simply defective or non-defective with respect to a given requirement or set of require- 
ments. Variables Inspection, on the other hand, takes into account the degree of conformance 
or non-conformance of the unit by measuring the unit of product along a continuous numerical 
scale and by describing its quality in terms of its position along that scale. 


There is, however, one distinct difference between administering the quality assurance of 
non-numerical quality characteristics inspected through the attributes inspection technique and 
administering assurance of measurable characteristics. That difference is the greater empha- 
sis that must be placed on the proper selection and training, and on the quality and closeness of 
supervision of personnel who are responsible for evaluating and making decisions on non-meas- 
urable quality characteristics. While this difference is mainly one of degree, it is nevertheless 
of fundamental importance, The distinction, of course, is caused by the dual role that the in- 
Spector must play simultaneously as testing instrument and judge. For as has been pointed out 
earlier, the evaluation of conformance to established standards is necessarily to a great degree 
subjective and interpretive. And regardless of how carefully standard samples have been se- 
lected, disagreement of interpretation as to the degree of a product's conformance to standard 
will still rear its ugly head in marginal cases. 


1, Selection and Training of Personnel 


As in all problems of assurance of quality conformance, the sine qua non selection of in- 


spectors having appropriate aptitudes for the job, followed by education, training and on-the-job 
supervision. In the case of quality assurance of non-numerical characteristics, however, 
certain traits are particularly critical due to the fact that the inspectors must judge conformance 
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subjectively through their senses rather than with physical, chemical or electrical measuring 
instruments or gauges. The Quality Control supervisor, when he hires personnel for this type 
of work, is in reality employing in one person both an inspector and a measuring instrument - 
namely, his sense of touch, color discrimination, sense of smell, etc, We will not concern 
ourselves here with a general discussion of what one should look for in a candidate for an in- 
spector's position, such as integrity, intelligence and appropriate physica! aptitudes, It might 
be well, though, to discuss briefly some of the special attributes that inspectors of non-numeri- 
cal characteristics must possess, 


First, personnel must be selected who have not only the necessary aptitudes for perform- 
ing the test method to begin with, but also the required acuteness of the sensory organs which 
will in effect be the testing instruments, Inspectors can, of course, be tested and screened for 
this aptitude. Second, candidates must possess the intelligence and judgement necessary to en- 
able them to translate effectively and consistently what their senses tell them into decisions as 
to conformance or non-conformance to the standards, Screening applicants for this trait is 
slightly more difficult, and chances of errors in the screening process far greater, Third, they 
must possess evenness of temperament so that outside, or inside, pressures will not affect their 
judgement - or to put it another way, throw their senses out of kilter, It is one thing to require 
a temperament that will make a man decisive and not falter when he has a gauge to back him up; 
it is quite another when he must defend and control his conviction that he is in control of his 
senses! This quality is essential to minimize the effect of a "bad day" on an inspector's judge- 
ment, to say nothing of the effect of his reaction to the pressures of irate operators whose work 
he is inspecting or to the many other pressures of production. It is all too easy for him to let ‘ 
his "measuring instrument" get out of "calibration" in such cases, And it requires a keen super- 7 
visor to know when an inspector is yielding to such pressures. This quality is difficult to pre- 
dict in an applicant, but adequate supervision will quickly detect it on the production floor. 


Having selected personnel possessing the above essential traits, and having indoctrinated 
them into company, departmental and job routines, the next step is to train them in the use of 
their senses by exposing them to all the shades of variations in product quality - to calibrate 
them, so to speak, But this alone is not enough, For even though their senses have been devel- 
oped to the point where they can compare products with the standard correctly, it is further es- 
sential to train them in such a way as to minimize the danger of “drift') This latter phenomenon 
is a well-known problem in this area of quality control, but will be defined in detail later. 


Perhaps the best way to avoid drift is to make sure that every inspector understands the 
importance of his decisions from the viewpoints of not only outgoing quality but also of the costs 
and delays caused by rework and scrap. This sounds trite and obvious, but I state it here be- 
cause, although such education is of basic importance in the training of any spector, it is espec- 
ially essential for inspectors responsible for assurance of non-measurable quality. Orientation 
in this area can be of inestimable aid in helping them "stick to their guns" when the going gets 
rough, Not only is such initial training necessary, but also continual follow up through reorien- 
tation conferences is vital to say nothing of their being called in on the details of quality com- 
plaints. Exposure to customer complaints will not only help maintain in them this critical feel- 
ing of responsibility, but also it will help give inspectors an added "feel" for their job and will 
tune their senses more keenly to their task. 


2. Administrative Problems 


As in the case of personnel selection the administration of the quality assurance activity 
involving non- measurable characteristics is similar to most quality assurance programs, There 
are, however, certain problems peculiar to our case, Again we will consider the problem from 
the standpoint of the floor inspectors and the quality laboratory, but the points brought out here 
apply equally to all phases of administration. One of the problems that is often difficult to de- 
tect is the natural tendency for an inspector's judgement, or even that of one entire department, 
to drift either toward an excessively tight interpretation or, conversely, towards a lax inter- 
pretation of the standards in the case of marginal products, This problem will be minimized but 
not completely eliminated by the use of tolerance standards as was discussed earlier. Some, 
from over-zealousness, will tend to tighten up their interpretation and demand perfection; others 
because of sympathy, say, for a production departments’ quality dilemna, will stray towards 
laxity. This natural tendency will be accentuated one way or the other by organizational indepen- 
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dence of the quality control department on the one hand or by domination of it by production on 
the other, But in any event it is an ever-present danger, the existance of which higher manage- 
ment must be continually aware, One answer to this problem is a periodic "audit" of the in- 
spector's judgement by properly oriented individuals, whose duty it is to assess the inspector's 
decision of border-line cases. Another solution is to check an inspector's judgement by having 
him pass on samples which have been previously compared and rated against the standards, Re- 
orienting the inspector will then be a relatively easy matter, 


One of the more insidious problems that may arise in administering such a program is the 
effect on an inspector's morale if his decision on border-line cases is countermanded, This may 
happen from time to time for a variety of reasons - either because the chief inspector may disa- 
gree with him or because the sales department may decide that quality may be more lax for a 
certain customer, etc. In any case, the effect on the inspector's morale may cause him per- 
manently to shift his interpretation of questionable cases, This is not so true with measurable 
characteristics where it is possible to waive products failing the standard by a certain fraction 
or percentage. But how does one define the degree to which the quality of a non-numerical 
characteristic may fail to match the minimum standard and still be waived? The only answer, 
of course, is to set up a new standard for each such case, Failing that, extreme care must be 
taken to explain to the inspector why his judgement has been overruled. Here, again, is an ex- 
ample which shows the rewards of orienting the inspectors to customers’ needs and complaints. 
The better an inspector understands why he is asked to vary his interpretation of what he can 
accept in such special cases, the less likely it is that he will become discouraged or let his 
judgement drift. 


The relationships between inspectors and production operators is another tricky problem, 
where through the inspector's authority to accept or reject work he in effect has a hand in 
writing an operator's pay check where the operator is paid only for good work produced. This, 
of course, is merely an on-the-firing-line extension of the problems of relationship between 
quality control supervision and production supervision. Once again the point at issue is the sub- 
jectivity of quality decision, One can easily imagine the pressures imposed on inspectors by 
irate operators who will be so penalized for rejected material. Obviously one of quality control 
supervision's most important administrative jobs is to orient production supervision and in turn 
help them indoctrinate line operators in the necessity of accepting the quality control depart- 
ment's decision, And it requires the understanding and active support of higher management to 
maintain quality control's authority and prestige by backing their decisions and educating pro- 
duction in this area, 


SUMMARY 


Control of the quality of non-numerical product characteristics, then, differs from the 
control of measurable characteristics in that greater reliance must be placed on the individual- 
on his ability to evaluate quality conformance accurately and consistently-for it is necessary to 
rely completely on his senses and on his translation of what his senses tell him into quality 
decisions, 


Through the use of intelligent screening and, even more important, through close attention 
to supervision of both inspection and production personnel, however, perfectly satisfactory re- 
sults should be attained, Nevertheless, the subjectivity of the quality evaluation method, is a 
basic weakness and this technique of quality control should be replaced by the use of numerical 
standards wherever it is possible and economically feasible to do so. 


STATISTICAL AIDS TO VISUAL INSPECTION 


C. Kenneth Divers, Supervisor of Quality Control 
Brown-Live-Chanin Division, General Motors Corporation 
Syracuse, New York 


This paper discusses some of the statistical methods that are available to the 
quality control.engineer for use in evaluating and increasine the efficiency or 
effectiveness of visual inspection, 


SAMPLING PLANS 


Sampline plans, when applied to inspection done in the plant between processing 
operations, provide an effective means of reducing inspection costs and improving 
quality. When applied to visual inspection areas, they have the added advantage of 
reducing inspection fatigue, esvecially that associated with hichly polished parts. 
As a result, better quality is passed with fewer scrap or salvage pieces. 


Inspection Procedures 


Die cast parts, after polishing, are assembled by the polisher into lots of 
various sizes on carts especially designed for that purpose. A typical cart is shown 
in Figure I with the part shown in Figure 2. If the part is produced on an automatic 
polishing machine, the unloader assembles the parts into lots. These parts are then 
randomly sampled by an inspector, Random sampling is assured by having the inspector 
use a deck of numbered cards that are thoroughly shuffled before drawing cards. The 
first card drawn designates the column that the part is to be selected from with the 
second card designating the row. 


A reiected lot is sorted 100 per cent by the polisher, the "bad" parts are re- 
polished or scrapped, and the lot is resubmitted to the inspector for resampling. 


The selection of the inspectors and the inspection standard is of utmost impor- 
tance in anv visual inspection scheme, We are not gacing a part or classifying it 
by go, no-go gages, but are visually inspecting it. Human judgment thus plavs a very 
important role, 


The quality standards used must be carefully chosen by top level inspection 
Personnel. These standards are then prominently displayed where the inspector has 
ready access to them and are used as a basis for acceptance or rejection decisions. 


However, the competence of the inspector, who samples and checks the parts, 
remains the dominating factor. This inspector must be conscientious, work with a 
minimum amount of supervision, and have consistency, or repeatability. of decisions. 
It is essential that he continue, based on the quality standard, to call a good part 
good and a bad part bad. To assure this, we have had the sampling inspectors rank 
ten carefully chosen parts in order of the degree of defectiveness. The repeatability 
of the inspectors' judgement has been excellent. The majority of the inspectors 
have worked on visual inspection assignments for a minimum of ten years. We have 
noted, however, in 100 per cent inspection areas, that the repeatability of decisions 
is comparatively poor. A part of X ouality will pass an inspection station in the 
morning and be rejected in the afternoon, or vice versa. Also in 100 per cent inspec- 
tion areas a gradual deterioration in quality will often not be noticed until a 
drastic change in quality level has taken place. We have not observed these pheno- 
mena in samoling areas. 
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Plans Used 


AOQL plans were chosen because it is essential that the parts leaving our pol- 
ishing department, in the long run, be of quality better than or equal to the AOQL. 
The quality of the plated part is directly dependent upon the quality of the polished 
part. 


The AOQL ranges that we have found practical run from 1 to 4 per cent. The 
chosen AOQL is dependent upon: 


(1) the budgeted scrap allowance, based upon scrap costs 
(2) the difficulty of polishing and plating the part; and 
(3) the end quality desired in the part, 


A typical sampling table used by the floor inspector is shown in Table I. The 
p', Values represent approximate incoming lot quality corresponding to the AOQL value 
on the AQQ curve for the plan. Values of the AOQL corresponding to p', can be com 
puted from tables of the Maximum values of (pa) (p'N) from given values of ¢ in acc- 
ordance with the formula 
AOOL= (pa) (P',N) where p' is of course the fraction defective of the incoming 
lot that gives the AOOL, 
i.e. for the plan N = 35,c #1 
max (pa) (p'N) = .8408 
AOQL = ,8408 = .024 = 2.4% 


35 
able 

Sample Reject 
Part Size Number AOQL (%) P'm (#) 
A 35 2 2.4 5 
BR 35 2 2.4 5 
Cc 25 1 1.5 4 
D 18 2 1.0 6 
E 75 4 2.6 4 


Advantages Of Sampling Plans 


The use of samplins plans instead of 100 per cent inspection has the following 
advantages: 


1. Inspection efficiency is increased 

2. The amount of inspection labor is decreased 

3. The quality level increases 

4, The emphasis on quality is placed upon production 
5. Rework and scrap are decreased 


As the inspection efficiency is increased, fewer bad parts are sent to the next 
process and, as a result, better quality parts with less scrap emerge from the total 
process, An inspector, after looking at 1000 or more parts that are highly polished, 
becomes fatigued and his eyes blur over defects. 


A decrease in the amount of inspection labor is a definite monetary savings 
that also comes as a direct result of the use of sampling plans. The following ex- 
ample will illustrate savings that we made in this area alone. 


Five hundred parts per hour were being produced by four different polishing 
lines, each line polishing a different part. Under 100 per cent inspection this re- 
quired four inspectors. These parts, however, are now placed in specially designed 
containers in lots of 500, The sampling inspector takes a maximum of 15 minutes to 
sample and check the skid if it is to be accepted; less time, normally, if it is 
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rejected. Thus, this one inspector can now judge the same volume of product that for- 
merly reauired four, 


Emphasis on quality is placed upon production through the use of sampling plans. 
In any 100 per cent inspection scheme, the burden of maintaining the quality of the 
parts beine insvected rests on the inspection department. As a result, the producer 
of the part (in this case the polisher) takes little pride in the quality of his work, 
for he knows that if it is not acceptable it will be sent back to him, Thus, the em- 
phasis on quality shifts from the producer to the inspector, 


Sampling plans, however, reject the work by lots if it does not appear to meet 
the prescribed quality level. It is now, not inspection's job to maintain a specified 
_ quality level, but production's, as the polisher responsible for a rejected lot must 
reinspect his work and bring it up to the quality level, either by replacing the bad 
parts with good ones or by repairing the tad ones. As a result, the production man can 
not make his quota if he has many lots rejected. 


Rework and scrap are decreased because better quality parts come from the pro- 
duction department. When parts are being 100 per cent inspected, they are accepted by 
the inspection department and production flow is kept steady regardless of the quality 
level. However, the use of sampling plans may result in an interruption in this pro- 
duction flow, This condition will occur when the average quality level submitted to 
inspection is poorer than the AOQL. Thus, a poor quality level is forcibly brought to 
management's attention, and either the cause of the poor quality will be eliminated, 
or 100 per cent inspection must be returned to the job, at greater cost. 


RANKING PROCEDURES 


The ranking process is a method that enables the user to assess the agreement of 
two or more individuals in judging the comparitive qualities of two or more items, 
This is the type of act that is performed when evaluating the merits of various cuts 
of steaks, bottles of whiskey, sharpness of cheese, etc. It is used by us in estab- 
lishing quality standards on buffed and plated parts. 


The procedure used entails the selection of ten carefully chosen parts that rep- 
resent the varying decrees of quality to be expected from a process. These parts, 
after being assigned numbers, are then ranked by the staff management members of the 
Quality Department. The overall correlation coefficient is computed and assuming the 
correlation is significant the parts are reassigned a quality rank, The quality stan- 
dard is then chosen from these parts, and is of course dependent upon the customers‘ 
quality requirements, The following example will show the basic computations involved. 


TABLE II 
Ranked 
Inspector Best ------------ -Worst 
Cc: 3832 @ 
D 1&5 6 8 7 3 2 9 «10 
Totals 6 


24 30 29 34 24 31 38 54 60 $330 
Compute the "Coefficient of Concordance" W 


W=s 
s -n 
max. 12 


where 


d - difference between the observed rank total and the exnected rank total 
assuming no difference in quality or judgement 
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m = number of inspectors 
n = number of items 


w= 21 (33-24)" 0)” 
(0? ~ 10) 
2 
=.71 
2970 


The simplest significance testing of W is done using Snedecor's F 
Compute a corrected W using 


w=s-] = 206-1 = 2115 - .71 
= = 2) $2 2970 42 2972 


for this case there is no difference 


Compute F as 

Fe (m-1) (5-1) = 2.84 
1-W 1-.71 229 

F = 9.79 


Degrees of freedom in the numerator = (n-1)-2 = ag = 8.67 or 9 
nm 
Degrees of freedom in the denominator 


n 

Then using the F table 

Sf level F= 2.10 14% level F = 2.84 


As the computed value of F is larger than the tabled value at the 1% level of signi- 
ficance the rankers exhibit a significant degree of agreement, However had the rank- 
ers failed to agree this would have resulted in a reselection of parts, Visual in- 
spection of the rank table shows that the judgement of the rankers was poorest where 
the quality of the part was "average". This is to say that it is difficult to decide 
on so-called "borderline" quality parts. 


AVERAGE QUALITY LEVEL CHART 


The percent defective chart has long been used as a means of controlling the 
"goodness" or "badness" of parts upon which the characteristic checked could not be 
measured but only classified as good or bad. In the application of the P chart to 
the control of die cast quality in our plant it has not been unusual to continually 
run a job at 8% defective ana periodically have complaints from our polishing depart- 
ment that the quality has decreased and upon recheck show the process still running 
8% bad, but the "bad" parts have become exceptionally bad in regards to quality. That 
is to say the degree of defectiveness has increased and consequently it has taken more 
labor and material to make acceptable the defective parts, 


Thus, a need was present for a more sensitive tool than the P chart. As a 
result of this the Average Quality Level chart was developed. 


atio: 


The first step is to pick out, preferably by means of a visual ranking technique, 
the Average Quality Level part; this part is assigned the number 0, The next part 
selected from the process should represent the best auality obtainable and this part is 
assigned the number #2, The next part, whose quality level is halfway between the 0 
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and # 2 parts is assigned the number 41, The worst quality part obtainable is then 
selected and given the number - 2. Finally, the part whose quality fails halfway 
between 0 and = 2 is selected and assigned the number - 1, We have thus selected and 
graded, in units of one, all the possible quality parts according to the process under 
PRESENT OPERATING CONDITIONS, These quality level standard parts are then retained so 
that all inspection personnel have ready access to these standards. 


Ten random parts are then selected every two hours (this time interval is at the 
user's discretion) and noted as 4 2, 41, 0, - 1, or = 2 for each part. The values 
thus obtained are averaged, keeping the sign and the average quality level thus record- 
ed and plotted on the average chart, 


Theory 


By assigning the numbers from - 2 to 4-2 to the five possible quality level 
pieces emanating from the process, we have in essence allowed ourselves to check the 
visual quality of the part by variables. 


The upper and lower quality level control limits (UQLL & LQLL) are expressed in 
terms of the standard deviation of the average ( oz de 
Qa 


WIL = - 3 
CS OF 


Now, the standard deviation of our parent population is 2/3, assuming that 6 Q’encom- 
pass 100%, not 99.73% of our values, The standard deviation of the average is O~ 
which for this case eauals 2/3 + fi which eouals .2108 or ,211 4 


For our case where the sample size is always 10, 


Thus, three times the standard error of the mean is .633. Accordingly, if two 
points in close succession fall outside the upper or lower quality level limit, we 
know the population mean has changed, This means that the average quality level part 
or 0 part is no longer the true average quality level part but that this part should 
be re-selected along with the other standard parts. 


The following examples in Table III will show the sensitiveness of Quality Level 
Chart in comparison with the P chart. 


TABLE III 

Example 1 

P = 50% 

Example 2 

Ten samples 1, 1, 0, 1, 2, -1, -1, <1, -l, <1, @=0 
P = 50% 

Example 3 

Ten samples 1, 0, -l, 0, 0, 1, <1, -l, <1, -1, @ = -.3 
P = 50% 

Example 4 

fen samples 2, 2, 1, 1, 1, <1, <1, -l, -l, 
P=5 


In these four examples a decided shift in degree of defectiveness has occurred 
while the process has remained 50% defective. This technique has proved most valuable 
on processes producing parts where this shift occurs, 
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OTHER QUALITY RATING SYSTEMS USED 


In addition to the Quality Level system we are using as a general control check 
the chart for defects per piece and to accurately assess the main difficulties or de- 
fects obtained in first run castings the critical-Major-Minor or Average Defect Point 
system is used, A typical check sheet for the latter method is shown in Figure 3 and 
is used in evaluating the quality of die castings after they have been polished. 


INSPECTOR EFFECTIVENESS 


The effectiveness or efficiency of the inspectors is determined using the well- 
known analysis of variance when comparing two or more inspectors or if comparing one 
inspector's results to a standard set of prior ranked parts the ranking techniaue can 
be used, 


An evaluation of three inspectors checking the casting quality of a header bar 
was made by having the men rate, using the Quality Level system, 26 subgroups of 5 
parts in each subgroup. The analysis of variance table is shown in Table IV, 


TABLE IV 
Analysis of Variance 

Source Sum of Squares Derrees of Freedom Mean Square 
Between Insp, 4,33 2 2.167 
Within Insp. 14.50 75 2193 
Total 18,83 77 0245 

F = Between Insp, Mean Sq. = 2,167 # 11,22 

Within Insp. Mean Sq. 2193 


Table value of F - 4.9, As this value is larger than the table value at the 1% level 
of significance, we conclude that the between inspector variability is significant. 
The inspector differences may then be compared using a linear contrast technique and 
judgement made accordingly. 


The following example will illustrate the use of a ranking technique when the 
"true" ranked order of parts is known and it is wished to assess an individual's 
judgement. 


True Rank a3 
Inspector's Rank 16 


990 990 


P= Spearman's rank correlation coefficient. 


d = difference between the true rank and the inspector's rank, 
n = number of items ranked. 


Students t = n2 = 237 = = 237 9.28 t =1.12 
fYi- 1 = (.37)2 } 


At the 1% level the table value is 3.555. A value as large as 1.397 could have 
occurred by change 20% of the time so this inspector has poor discrimination powers. 


Inspectors variability evaluation methods of this type have several important 
advantares: 


8 
= = 1 - 6 (74 ye £32 
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1. The best inspectors can be determined and they can be placed in the critical ins- 
pection areas. 


2. The poorest inspectors can be replaced or they can be assigned duties in less 
critical areas. 


3. If the within inspector variability is large, studies can be made to determine 
the reasons. 


4, Management's rating of the individual inspectors is put on a scientific basis. 
CONCLUSION 


It is hoped that some of the basic viewpoints and ideas covered in this paper 
will prove of aid to you in visual inspection areas, They have been of significant 
help to the Brown-Lipe-Chapin Division of General Motors Corporation in our visual 
inspection areas. The writer feels that the application of statistical methods to 
visual inspection areas has only begun and is vitally interested in other investi- 
gators efforts in this field. 


1. Bowker, A. H. and Lieberman, G. J. Hangbook of Industrial Statistics, Prentice- 
Hall, Inc., Englewood Cliffs, N.J., 1956. 


2. Duncan, A. J. Quality Control and Industrial Statistics, Richard D. Irwin, Inc., 
Homewood, Illinois, 1955. 


3. Kendall, M. G., The Advanced Theory of Statistics, Volume 1, Fifth Edition, 
Hafner Publishing Company, N.Y., 1952. 


4, Moroney, M. J., Facts from Figures, Penquin Books Lt'd., Harmondsworth, Middle- 
sex, 1953. 
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VALUE ANALYSIS AND ENGINEERING 


L. D. Miles, Manager - Value Service 7 
General Electric Company, Schenectady, New York 


QUALITY CONTROL AND VALUE ANALYSIS k 


As the technology of Quality Control rose from economic need, so has the grow- 
ing technology of Value Analysis. 


As the objective, in very general terms, of the Quality Control Program has 
been to assure that the product accomplishes the functions the customer wants--well 
and reliably, so the general objective of Value Analysis is to assure that the functions 
which the customer wants are accomplished at appropriate costs. 


In preparing for this discussion with you, I asked a sampling of people two 
questions: 


1. What do you mean when you say a product has good quality? 
2. What do you mean when you say a product has good value? 


The consensus of the answers was this: 
1. If a product... 


"does what I expect it to 
as long as I expect it to... 
it has good quality." 


2. If a product... 


"does what I expect it to 
as long as I expect it to... 
and - costs what I think it should... 
it has good value." 


These simple statements will confirm the more sophisticated charter which 
started the research work resulting in the techniques of Value Analysis a few years 
ago. 


The objective was to develop techniques which would keep quality, safety factors 
and customer features but more efficiently and effectively provide appropriate costs. 


LOWERCOST AND BETTER QUALITY 


As the special techniques were developed, it was a gratifying surprise to find 
that they improved value two ways -- one by lowering costs, a second by improving 
quality. 


A pattern of techniques based upon product or service "function" developed. 
The Value Analysis process started with a clear understanding of the function or func - 
tions desired by the customer, separated and classified each, provided a system for 
generating alternative means for accomplishing the functions, means for establishing 
the dollar value of each, and techniques for overcoming the stoppers and "roadblocks" 
encountered. 
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It became very understandable that these techniques and the associated special 
knowledge which promoted a clear view of the desired functions, then provided alterna- 
tive solutions, not only bring forth for review solutions that are lower in cost, but also 
many that are better in quality, more simple, and more reliable. 


Experience in a recent Value Analysis technique-training course will be of spe- 
cial interest to Quality Control people. 


Training is of a teach-learn-do type. Seventy-five men in twenty-five groups 
of three, each had product assemblies or production problems to which they applied 
the Value Analysis techniques as they learned them. The projects of twenty-two of the 
groups left nothing to be desired in quality. They accomplished their functions well, 
constantly, and reliably--the only needed improvement was lower costs. To three 
of the groups, however, were provided projects which, although accomplishing the 
desired function, were sometimes troublesome. These men were challenged to use 
the techniques to eliminate the troublesome and unreliable features even to the extent, 
if needed, of adding $10,000 per year per project to the cost. All three completely 
ended the quality problems in their projects providing simpler, better, lower-cost 
means for accomplishing the functions and all three at lower costs. 


To provide basic understanding of this new technology, some orientation follows. 
VALUE ANALYSIS ORIENTATION 


Let's distinguish between two types of ''value"’, which I will call "use" value and 
"esteem" value. 


For example, the value of my necktie may be about $2. Now, let's separate the 
$2 into its parts of "use" and "esteem" value. The "use" value to me of this necktie 
is zero. It has no "use". Its "esteem" value is $2. I must wear it to conform to cus- 
tom. If I didn't I would suffer the loss of more than $2-worth of "esteem". Asa 
comparison, this tie clip with a value of $2 has a "use" function and an "esteem" func- 
tion. The "use" function--what the tie clip does--could be reliably accomplished by a 
suitable pin or other simple product that would cost not over one cent. Therefore, we 
can roughly say its "use" value is one cent and its "esteem" value $1.99. How about 
the button on the shirt? If it has a total value of one cent, probably nine-tenths of it 
is "use" value and one-tenth is "esteem" or "appearance" value. A nail, for instance, 
has an even higher proportion of "use" value, approaching one hundred percent. 


The Value Analysis techniques provide means for establishing the dollar value 
of the "use" and of the "esteem" portions of the function. 


Now let's take a quick look at the so-called relationship between dollar value 
and cost. The value of this clock is $5. This we know because we evaluate by com- 
parison; we have seen them around, we have bought them. Because we know what it 
costs, we have given it a value. 


Now, let me add more cost to it. I am going to do some work on this clock, 
adding labor and overhead to it. Do you think that the added cost of the modification 
will increase the value? Watch carefully! (Smashes clock on floor.) What is the vale 
of the clock now? It's zero, or scrap value, although its cost was increased, by labor 
and overhead. 


Two things we learn from this demonstration: 


1. There is positively no relation between cost and value. The cost has been 
increased, but the value decreased. 


2. A sound usable base for value is tunction. The function capability of that 
clock changed! 


| 
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Let us, for the time being, use this definition of value. ‘Value is the lowest 
cost to reliably accomplish a function''--where function consists of those elements 
which cause the product to "work" and those elements which cause a product to "sell" 


Some of the approaches and techniques which Value Analysis uses are: 


1. Approach--Clearly and sharply identify and understand the function, begin- 


assemblies, then of the subassemblies, etc. 


ning with the function of the overall product, breaking it down into the functions of 


2. Technique--Evaluate this function; i.e., assign a dollar and cents figure to 


it which is the lowest cost to reliably accomplish the required function. 


This will be 


determined by a creative search for engineering, manufacturing, and other value al- 
ternatives which would reliably accomplish the total function together with the overall 


costs involved. Obviously, this evaluation will be just as good as the tools and knowl- 
edge and effectiveness of the evaluator. For example, for the function of containing 

) 200 gallons of gasoline in a landing craft which has a useful life of eight years, what 
is the value? Four 50-gallon drums might cost a total of $25, but probably they 


wouldn't stand the environment. They would need some sort of coatings. 


As a first 


| guess, let's estimate that the coatings would slightly more than double the cost. 

| Estimate -- $60. Now we have a quick estimated value of $60 for the function. Al- 
ways do this before finding out how the job is planned to be done, because it will lead 
to new and startlingly simple, reliable, and lowest-cost solutions. In this case, the 


specification actually called for specially-fabricated, special alloy tanks costing $520 
each. The result was that in this procurement of tanks for 1000 ships, the cost to you 


and me, the taxpayers, instead of being the expected $520,000 was $80,000. ..and the 
function was indeed accomplished by using four drums with appropriate coatings just 


as on the preliminary evaluations, for $80 each set. 


3. Another and different technique is called, "Blast, Create--then Refine." 
This means, get a clear mental picture of the function that is required and of the way 
it is planned to accomplish it. Then mentally blast this down to something that will 
have only a small fraction of the cost and will only partially accomplish the function 


or will have some of the attributes needed in the finished product. Next, 


| fine, adding increments of performance, of function, together with their 
of cost, until the product now will accomplish the total job with adequate 


creatively re— 
increments 
reliability. 


This approach sounds simple, but it is amazingly effective. For example, in the gas 
tanks just used. ..first was the blast to the $25 drums which would not accomplish the 
total function; next, came a review of what must be done to provide the total function. 
Then followed the vital, but simple solution--coat the drums--which added only $55. 
So now, by the use of this step-by-step technique, the total function is provided with 


the same reliability for $80 instead of $520. 


Eleven additional techniques are learned by the value engineer, each functioning 


value alternatives. 


mize this problem calls for lots of understanding, forethought and care. 


is to save time, not to save money. But before the production design is 


in its own type of simation to cause the development of applicable reliable low-cost 


One of the major problems in obtaining acceptance of Value Analysis is that men 
feel toward the value part of their own work as they would feel toward their artistic 
work in painting a picture. They have done the best they know how; there are no 
measurements as in "performance" engineering to tell them how good or bad their vahe 
is. They feel very emotional, edgy. The thought of anyone making suggestions which 
affect the value part of their job gives them the same emotions which a painter feels 
| when he hears what he believes to be "untalented people" criticizing his art. To mini- 


Another major problem is the proper timing of Value Analysis effort. In re- 
search and development activities, top-grade "value" work is of great importance but 
has not been generally so recognized. Research and development work is substantially 
performance oriented. Feasibility models are made as fast as possible--the problem 


released for 
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quantity manufacture, it needs a large contribution from cost-centered, ''value''- 
oriented engineers. It is too often the case that pressure oftime and shortage of value- 
oriented capabilities forces the subsequent quantity manufacture of ''development mod- 
els", a procedure creating problems in complexity, very high cost, extra weight, and 
poor reliability. In order to gain time, the value work can be done in the laboratory 
simultaneously with the performance work. 


SUMMARY 


The technology of value analysis and value engineering consists of a system of 
techniques and of special knowledge which are used in the design concept stage, the 
design stage, the purchasing stage or the manufacturing stage to efficiently identify 
unnecessary costs so that they may be prevented or removed. 


This technology is function based and operates to create alternative means for 
reliably accomplishing functions. By providing better answers, not only costs, but 
also quality and reliability are improved. 


Quality control makes its major contribution to value by aiding the customer to 
secure appropriate, reliable and continuing performance. 


Value analysis and engineering make their major contribution to value by aiding 
him to secure appropriate costs. 


| 
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ORGANIZATION OF THE INSPECTION FUNCTION FOR MAXIMUM EFFECTIVENESS 


Everett H. Dale, Quality Manager, Aeronautical Division 
Minneapolis -Honeywell Regulator Co., Minneapolis, Minn. 


INTRODUCTION 


How effective is your inspection function? Is it effective enough? Does your department 
meet the requirements of its assigned objectives? Have you asked yourself these questions 
lately? You have! Chances are you have already discovered that the organization structure of 
this portion of your department is important to its effectiveness. Take a look at your competi- 
tors or look at other departments within your own plant. Do you see some departments oper- 
ating more effectively than others? There is a good probability that a direct relationship exists 
between effectiveness and organizational structure. 


Now if you have the opinion that organizational structure is a cookbook thing - you're dead 
wrong. The organizational structure you need for your department can't be found on page 275 of F 
some handbook on organizational structures. Organizational structures need to be tailored to the 
specific objectives of your department. Good, effective inspection functions don't just happen. 
They result from careful and thoughtful planning. 


LOST? If you are, a little study of the problem will help to clear the air. In this paper, 
we are going to discuss the generally accepted principles essential to the effective organization 
of your department. In addition, we will illustrate the reorganization processes, by showing the 
step-by-step progress made by the inspection function of Honeywell's Aeronautical Division at 
Minneapolis in establishing an effective organizational structure for its quality assurance effort. 


THE BASIC PRINCIPLES OF ORGANIZATION 


Once the job of organizing or reorganizing begins, a study of the task turns up a system - 
atic approach for the formulation of organization structure. This systematic approach recognizes 
the existence of a set of basic principles of organization. Employment of these principles in 
planning the organization structure generally results in efficiency and effectiveness; neglecting 
these principles, on the other hand, most often promotes deficiencies in the work of management. 


This set of principles of organization must associate success with a dual achievement. It 
must provide for an effective, clear subdivision of management responsibilities and at the same 
time maintain an integrated unity in action. 


F. W. Taylor, H. L. Gantt, H. Foyal and other early proponents of scientific manage- 
ment began to recognize the existence of specific basic principles of organization for industry 
late in the nineteenth century. This was preceded by many examples of definitive organization 
structure as evidenced by the organization of the Roman Legions, the Roman Catholic Church and 
most early governments. 


However, long before this, we have evidence that some formal organization and delegation 
of authority and responsibility was necessary to achieve efficiency and effectiveness and to attain 
predetermined objectives. The most striking example appears in a passage in the Old Testament, 
EXODUS 18:13-24. In this passage Moses was faltering under the load of work in planning the 
mass exit of his people from the land of Egypt. He was advised py his father-in-law to “select 
able men from among his people and place such over them, to be rulers of thousands, rulers of 
hundreds, rulers of fifties, and rulers of tens". Moses took this advice and the able men 
"judged the people at all seasons; the hard causes they brought to Moses, but every small matter 
they judged themselves". 


This biblical passage clearly contains many of the basic principles of organization used by 
today's management consultants to rejuvenate faltering and ineffective enterprises. 


300: 70:438 
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Taylor's early work contributed most to establishing the first listing of basic principles 
of organization for industry. Since that day, additional principles have been added with many 
modern experts on the subject agreeing and disagreeing with the definition of each principle. All, 
however, agree that a set of basic principles of organization govern the effectiveness of each 
enterprise. 


Lyndall F. Urwick, chairman of Urwick, Orr and Partners, Ltd., London, and a con- 
consulting specialist of considerable reknown in modern organization and management circles - 
clearly summarized the principles acceptable to most of the students of the field. These are as 
follows: 


PRINCIPLES OF ORGANIZATION 


OBJECTIVE Every organization and every part of every organization 
must be an expression of the purpose of the undertaking 
concerned or it is meaningless and therefore redundant. 


SPECIALIZATION The activities of every member of any organized group 
should be confined, as far as possible, to the per- 
formance of a single function. 


COORDINATION The purpose of organizing per se, as distinguished from 
the purpose of the undertaking, is to facilitate coordi- 
nation; unity of effort. 


SPAN OF CONTROL No person should supervise more than five, or at most, 
six, direct subordinates whose work interlocks. 


BALANCE It is essential that the various units of an organization 
should be kept in balance. 


DEFINITION The content of each position, both the duties involved, 
the authority and responsibility contemplated and the 
relationships with other positions should be clearly de- 
fined in writing and published to all concerned. 


AUTHORITY In every organized group the supreme authority must 
rest somewhere. There should be a clear line of au- 
thority from the supreme authority to every individual 
in the group. 


RESPONSIBILITY The responsibility of the superior for the acts of his 
subordinate is absolute. 


CORRESPONDENCE In every position the responsibility and the authority 
should correspond. 


CONTINUITY REORGANIZATION is a continuous process; in every 
undertaking specific provision should be made for it. 


PLANNING THE ORGANIZATION STRUCTURE FOR AN INSPECTION FUNCTION 


In 1955, the management of the Inspection Department at Honeywell's Aeronautical Di- 
vision recognized that its organization structure had not kept pace with the division's overall 
growth. The Aeronautical Division had grown in a relatively few years from a small Special 
Products Group into an independent division of approximately 6,000 employees. Furthermore, it 
was recognized that the complexity of the division's product line and the volume of growth would 
continue. At this time, the management of the Inspection Department decided to make a thorough 
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study of its organization structure, its purpose, and its management. 


The discussion which follows illustrates the process of reorganization. Care was taken 
to describe the specific steps and relate them to each principle of organization as defined in pre- 
ceding paragraphs. Figure | is an organization chart representing the organization in the years 
prior to the study. 

ABRO INSPECTION ORGANIZATION CHART 
MINNEAPOLIS - HONEYWELL 
1950 


PRODUCTION MANAGER J 


= CHIEF INSPECTOR ] 


SPECIAL MANUFAC RECEIVING | | STATISTICA: TOOL 
PROJECT INSPECTION | |INSPECTION INSPECTION DESIGN INSPECTION <UALITY INSPECTION 
INSPECTION FOREMAN POREMAN POREMAN AND FOREMAN CONTROL FOREMAN 
FOREMAN BLDG. #2 BLDG. #1 CONTROL ENGINEERS Sik 
“SUBASSEMBLY FOREMAN CONPROL 
ASSISTANT ASSISTasT 
POREMEN FOREMAN 


FIGURE | -- Inspection Department Organization Prior to 1955-56 Study 


THE PRINCIPLE OF OBJECTIVE 


Don't try to develop a new organization structure until you have accurately de- 
fined the objective for the organization! Remember that each element of an 
enterprise must have a purpose or an objective in keeping with the objectives 
of the total enterprise. 


To this end, the Inspection Department, under its manager's direction, began a series of 
staff sessions over a period of one year in 1955-56 to determine and define the objective of the 
Inspection Department. Figure 2 represents the final output of these sessions. It was determinai 
that the division encountered twenty-one basic quality functions in pursuing its primary objectives 
as an integrated enterprise. Each of these functions were carefully defined at the conclusion of 
the staff sessions. 


The staff immediately realized that all quality functions were not necessarily the re - 
sponsibility of the Inspection Department. After tentatively establishing primary and secondary 
responsibilities for each quality function, they were submitted for review and criticism to the 
division's general manager and the department managers on his staff. The approved quality 
functions along with the definitions and assignments of responsibilities were distributed under the 
signature of the general manager. 


Selecting those quality functions where the Inspection Department held primary or second - 
ary responsibility provided the objective of the Inspection Department. This completed phase one 
of the reorganization. 
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QUALITY FUNCTION RESPONSIBILITIES aes age 


FIGURE 2 -- Distribution of Quality Function Responsibilities 


PRINCIPLE OF SPECIALIZATION 


Determine the functions for which your department has responsibility. Define 
these in detail. Organize in a manner limiting the responsibility of each 
member as far as possible to a single function. 


The next step taken in reorganization by the existing staff was to analyze each newly de- 
fined responsibility of the Inspection Department to determine the organization structure neces- 
sary to implement those functions assigned to it. It was immediately obvious that there was a 
need for two sections within the department - a “line” or implementing section and a "staff" or 
planning section. Evidence that this need existed in the past is illustrated by the following ex- 
tract from the minutes of an early staff discussion on reorganization: 


"In the present organization the Inspection Department has its “line” organi- 
zation rather clearly,laid out". . .""This section now has the execution function 
very definitely. However, the planning and service functions are scattered in 
several loosely connected and largely uncoordinated groups. In fact, much of 
the planning work is left to the line through default, i.e. no one has done any 
basic planning for their jobs until the job actually reaches them. They are 
forced to do the best they can under the pressures of the immediate situation." 


The reorganization discussions led to the agreement to divide the department into two 
sections based on specialization of basic activities - i.e. planning vs. implementation: 


@ INSPECTION 
@ QUALITY ENGINEERING 


(MINE APOUS- HONEYWELL 
= 
coment 
= J 
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At this point in time, recognizing the broadening scope of the Inspection Department's 
responsibility, division management agreed to a change in name, Quality Department, the 


chosen designation, included both of the new sections. 


PRINCIPLE OF COORDINATION 


Speciaiization promotes segregation of individual efforts. Provide in your 
organization structure some means of obtaining unity of effort. 


The new organization for the Quality Department was provided with two basic sections - 
Inspection and Quality Engineering. This action of division of effort for specialization permitted 
a clear distinction between the planning and implementing functions. Each section had its spe- 
cific responsibilities, but each had a clearly defined dependence on the other for continuation of 
its effort. Success of the department's effort depended upon both sections working together as a 
coordinated team. The coordination was attained by having both sections report to a quality 
manager. To promote more effective use of the inspection function, the quality manager was 
placed in a position reporting directly to the division general manager. Previously the chain of 
command had the quality manager, then the chief inspector, reporting to the director of pro- 


duction. 


PRINCIPLES OF AUTHORITY, RESPONSIBILITY AND CORRESPONDENCE 


Don't expect success if you are not ready to delegate sufficient authority to 
match the responsibility given. It is said that the definition of "hell" isdele- 
gation of a responsibility without corresponding authority. 


Honeywell's twenty-one quality function study served not only as a definition of objective 
but also clearly designated the distribution of responsibilities within the newly organized Quality 
Department. Figure 3 shows the quality function responsibilities delegated to each section of the 


department. 


INSPECTION SECTION 
PRIMARY RESPONSIBILITIES 


@ Inspection of Gages, Instru- 
mentation and Tooling 

@ Receiving Inspection of Pur- 
chased and Subcontracted 
Materiel 

@in-Process Acceptance In- 
spection of Manufactured 
and Assembled Product 

@ Control of Rejected Items 

@ Acceptance Inspection of 
Completed Products 


SECONDARY RESPONSIBILITIES 


© Determination of Vendor 
Quality Capability 

O Process Control During Manu- 
facturing and Assembling 
Operations 


QUALITY ENGINEERING SECTION 
PRIMARY RESPONSIBILITIES 


@ Collection and Distribution of Quali- 
ty Data for Analysis 

@ Establishing Parts, Assemblies and 
Device Acceptance Procedures 

@ Planning and Obtaining Gages and 
Instrumentation 

@ Coordination of Product Acceptance 
with the Customer 

@ Evaluation of Customer Quality 
Satisfaction 

@ Quality Audit of Finished Products 


SECONDARY RESPONSIBILITIES 


QAnalysis of Contract for Quality Re- 
quirements Specified 
OQ Evaluation of Quality Capabilities 
of Production Facilities 
OCorrective Action to Prevent Re- 
currence of Production Discrepancies 
OClassification of Quality Character - 
istics for Acceptance Purposes 


] 


FIGURE 3 -- Delegation of Quality Function Responsibilities in the Quality Department 
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The organization structure was completed using the quality function responsibilities as a 
guide for its requirements, Figure 4. A letter from the division general manager introduced the 
new organization structure. This provided a clear expression by management of the Quality De- 
partment's “line of authority" for accomplishing its quality function responsibilities. 


INSPECTION SECTION QUALITY ENGINEERING SECTION 


CHIEF 
R 


GROUND SUPPORT SHOP TOOL & GAGE INSP. INT. PARTS ¢PLATPORMS GAGE DESIGB 
IP. INT. PARTS INSP. FLIGHT CONTR eREC. INSP. INSTRUMENTATION 
©@ RECEIVING INSP. CONTROL © FLEX SHOP I INSTRUMENTS, DESIGN 
© PLIGHT CONTROL © HYDRAULICS © SUB CONTRACT © GUIDANCE ° GSE *GaGE & INSTR. 
INSTRUMESTS INSTRUMENT LAB © FINAL PART INSP. CONTROL MODEL SHOPS DEVELOPMEST 
PACKING & PLATFORMS HYDRAULICS 
SHIPPING © PLEX SHOP 


FIGURE 4 -- Quality Department Organization After 1955-56 Study 


PRINCIPLE OF DEFINITION 


With a definitive organization structure complete, each level of management and super - 
vision was defined. This included: definition of the duties of the position, the authority and re- 
sponsibility vested in the individual position, and the working relations between that position and 
the remainder of the organization. Position descriptions were prepared as necessary and pro- 
cedure manuals were published to amplify on the methods used to satisfy specific responsibilities. 
Today an up-to-date procedural manual "QUALITY AT HONEYWELL" outlines the functioning of 
the department. Additional implementation appears in detailed procedures for specific areas. 


PRINCIPLE OF SPAN OF CONTROL 


The new organization structure limited thc number of supervisors reporting into any one 
office to five. Each of these represent functions which essentially were related. In the earlier 
organization of the Inspection Department, as many as thirteen inspection areas reported to one 
inspection general foreman. Currently, the quality manager has two men responsible directly to 
him - one representing Inspection; the other with responsibility for Quality Engineering. In ad- 
dition, a small central staff was recently added to compensate and plan for changing responsi - 
bilities in the total quality function. 


PRINCIPLE OF BALANCE 


The reorganization provided a balanced structure for assignment of responsibilities in 
each section of the department. Within the Inspection Section, the assembly inspection group was 


vice 
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split, providing two inspection general foreman pcsitions to overcome an imbaiance in the earli- 
er organization structure. When finally completed, each of three inspection areas contained ap- 
proximately 100 people reporting into an inspection general foreman. 


PRINCIPLE OF CONTINUITY 


“The structure of organization which is best for a concern today will not be 
the best in all its details a month from today, if the concern has grown as 
any live concern would wish to. . . continuous reorganization can no more 
be left to chance --- or the ‘happening to think of it’ --- than can routine 
operation. Some kind of specific provision must be made for it or it will 
lag behind the needs. " 


In 1955, the Honeywell Aeronautical Division's Inspection Department was organization - 
ally falling behind its challenge. The structure had not been made flexible enough to meet the re- 
quirements of increasing product complexity and the changing requirements of its task. Today 
the organization is adjusting to the changes as they occur. The most recent example has been in 
the area of special project quality management. This is illustrated by a “miniature” Quality De- 
partment within a department as shown in Figure 5. Today's reorganizations are a simple task, 
because each is made as the necessity arises. 


DIVISION 
QUALITY 
MANAGER SPECIAL PROJECT 
MANAGER 
SPECIAL 
PROJECT 


QUALITY MANAGER 


| 
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INSPECTION «UALITY ENGINEERING 
QUALITY POREMAN SUPERVISOR 


INSPECTION ASS'T 
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INSTRUMENTATION 
ENGINEERING 


QUALITY CONTROL 
ENGINEERING 


RELIABILITY 
ENGINEERING 


INSPECTION ASS'T 
QUALITY POREMAN 


INSPECTION aSS'T 
QUALITY FOREMAN "Cc" 


FIGURE 5 -- Example of Special Project Quality Organization 


REORGANIZATION RESULTS 


HONEYWELL'S 1955-56 Inspection Department reorganization allowed specialization in 
its two basic areas of responsibility - quality planning and inspection of product quality. In- 
spection supervision, freed of its responsibility for last minute planning, began to assume its 
‘more important responsibilities - supervision of personnel and control of effort. Within a short 
time, inspection management was in a position to make use of the services of the division's staff 
functions to increase its efficiency and to improve its effectiveness. The following are examples 
of improved conditions: 


@ Labor relations improved as supervision began to devote more time to its prime re- 
sponsibility of supervision and work direction. 


a 
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@ Inspection supervisors free of the uncertainties of earlier experience of poor planning 
began to find time to attend management development training courses. Supervisors 
began to assume their proper role. 


@ Inspection supervision learned to make maximum use of the division's s‘aff functions: 
personnel, cost control, work measurement. 


@ Inspector qualifications were markedly improved by a joint effort with Personnel of 
developing special selection and testing techniques for hiring Inspection persovnel. 


@ Eighty per cent of the inspection positions now come under the special selection and 
testing program. Currently 45% of all inspectors are skilled technicians with 1200 - 
2400 hours of special trade school training or equivalent. 


@ TEMPO, an(Total Evaluation of Management and Production Output) internal work 
measurement program, has been introduced throughout the inspection area permitting 
an inspection supervisor to analyze his groups daily output for necessary improvements 
in efficiency and effectiveness. 


@ In the newly organized Quality Engineering Section, the organization was staffed to 
permit concentrated specialization in several quality planning areas. Specialists in 
device quality planning, statistical quality control, and gage and test equipment engi- 
neering provided the advancement of the device quality management concept and better 
engineered test equipment. The latter activity, alone in itself, promoted increased 
efficiency in the inspection function. 


@Today, because of the departments improved organization and increased efficiency, 
fewer inspection personnel are needed to do a higher volume of work at a considerably 
greater level of complexity. 


CONCLUSION 
A few words of suggestion are advisable before starting to reorganize: 


@ Don't reorganize until you have given the project sufficient consideration. Don't 
tamper with your organization with spur of the moment actions. 


@ Don't try to fit the organization to the people available. Whenever practical, people 
should, instead, be found to fit the needs of the organization. If you look carefully, 
you'll probably find most of the talents you need in your present organization. 


@ Make your organization structure clear. Fit the responsibilities expected to the 
authority given. 


@ Provide for specialization in every area possible. An individual works best when he 
has the greatest knowledge of his task. Make the inspection supervisor a supervisor, 
not a jack-of-all-trades and master of none. 


@Continually review your organization structure. As changes occur, keep the organi- 
zation up to the situation. 


Reorganization of your inspection function and strengthening its organization structure 
can prove to be an important factor in improving both efficiency and effectiveness.. Following 
the basic principles of organization, as outlined, offers a paved road to success but not a sure 
cure. Organizing is only one link in the chain of good management, and mauagement is getting 
results through people. If you want maximum strength in your department's management, forge 
strong links for the other six aspects of management. Provide for a strong organization, but 
also provide for proper forecasting, aggressive planning, capable commanding, careful 
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coordinating, detailed controlling and effective communicating. People are important; don't 
neglect them; provide them with the tools needed to get the job done right. 
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THE INSPECTION ENGINEERING APPROACH 


Norman J, Miller, Technical Advisor, Quality Assurance 
Engineering Division, Frankford Arsenal, Philadelphia, Pennsylvania 


I would like to discuss some of the Inspection Philosophies that have been 
developed in the Military Procurement Functions during the past several years, I 
refer to them as Inspection Philosophies because they represent a siginificant change 
in the approach to purchase of materiel from an inspection standpoint, When we talk 
of troubleeshooting procurement inspection we must, in the Military sense, think of 
it today in terms of Military Specifications, 


One of the significant changes in the approach to inspection of Military supplies 
has been the philosophy put forth in Department of Defense (DOD) Directives and 
Technical Service Regulations which establish responsibilities for inspection as 
being, "that the supplier shall be required to perform the examinations and tests set 
forth in specifications to substantiate conformance of supplies to specification 
requirements." The method of conveying the kind and amount of inspections to be 
conducted by the supplier represents another significant change in Inspection 
Philosophy. We now establish contractually in Section (4) of our Military Specifi- 
cation, a Quality Assurance system consisting of sampling plans, classification of 
defects, acceptable quality levels, test procedures, reference list of inspection 
equipment designs, and acceptance and rejection criteria, We also have established 
verification procedures to be used.by Government Inspectors to determine the adequacy 
of the supplier's inspection systems, This is considerably different than the 
previous approaches to inspection, where in many cases the autonomy of the Government 
Inspector with regard to "how" he was going to inspect, "what" he was going to 
inspect, etc., was maintained purely from a personal interpretation of technical 
requirements on his part. Certainiy the progress in Inspection and Quality Control 
diverted inspection efforts from a hererogeneous group of methods to something 
that has a well defined purpose, This has been brought about of course through the 
development of Quality Control, sampling theory, control charts, and other decision 
criteria which have acted as a catalyst in moulding inspection efforts into well 
defined purposes, So the question of trouble-shooting as we know it today, has as its 
basis the utilization of such documents as MIL-Q-9858, "Quality Control System 
Requirements", MIL-I-45208, "Military Specification Inspection Requirements, General 
Specification for", and other supporting documents relative to requirements for 
Inspection and Quality Control systems. These are the basis for trouble-shooting 
because they establish within the framework of the requirements, criteria of a 
contractual nature defining in some cases in broad terms, and in other cases in 
specific terms, what the Government desires in terms of Quality Assurance from the 
supplier. 


Quality Assurance in the procurement function is of course related to the Mili-e 
tary Specification, defining the product for service. So the essential elements then 
that constitute a good Quality Assurance system must be defined in detail, to support 
the technical requirements for the material or product for service, Here basically 
is a delineation of the Engineering approach to establishing a document for Quality 
Assurance with a proper perspective, 


Specifications in the system of supply function as vital components in the 
procurement, manufacture, inspection, and delivery phases of the Army system of 
supply. In the procurement phase, the specification must detail all the requirements 
of the item which will enable all bidders seeking an award to calculate costs on an 
equitable basis, from both the Government and Contractor point of view. If all 
requirements are clearly and accurately stated, with no ambiguity or omissions, all 
bids can be presumed to be based on the delivery of comparable items, under identical 
conditions of inspection and acceptance standards, 


From a supply and logistics standpoint, a specification is perhaps the most 
important document in the procurement function, We must think of this document as a 
repository of the necessary information to supply the using Forces with a materiel or 
product which they originally generated the need for. It has become a resting place 
of engineering effort in the form of drawings, reference documents, statistical 
techniques, and narrative descriptions of the skills and know-how of a great many 
people. It should be able to stand on its own in the procurement function as a 
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completely integrated document that will support the description of the product or 
service outlined in Section (1). 


Essential considerations in the design of the specification and its related 
engineering drawings and documents, are that they must convey the specific details 
necessary to manufacture an item, Quality Assurance requirements, including all the 
phases of inspection and test, must be clear and specific. The Contractor should and 
is entitled to know in advance how Quality Assurance will be effected, The Inspector 
must be given all the information, equipment design, and procedures necessary to 
assure that the item is acceptable. 


Technical and other qualitative terms may have several interpretations, or shades 
of meaning. Procurement contracts and the specification portions of these contracts, 
must be written in language clear and concise, so that it will convey the desire of 
the Government to prospective producers, Vagueness of meanings and implications of 
words and terms are vital. This is particularly important in the attitude of those 
engaged in the actual preparation of specifications, The legal aspects of procure- 
ment also are a part of the controls that must be borne in mind during the prepara- 
tion of a specification, In order to procure acceptable service and supplies, it is 
necessary that the buyer and the seller come to an agreement that will stipulate 
exactly what is desired. The contract and the specification should represent such an 
agreement. 


The Quality Assurance Provisions of specifications should accomplish their 
objective in a manner that is clear, concise, and technically adequate. The informa- 
tion should be clear as to intent, thereby avoiding contractual and legal controversy 
and providing for standardized interpretations by inspectors in different locations 
and Plants, 


The hazards to be avoided are over- or under-specification of quality assurance 
provisions, If over-specified, it might be impossible to procure the item, the price 
will be higher, the cost of inspection will be increased, and there are increased 
chances of Government and Contractor disagreement. If the provisions are incomplete 
or are under-specified, the item procured may not be as required. Some consideration 
in the basic structure of a quality assurance system are determined by:- 


1. Degree of assurance 

2. Inspection efficiency 

3. Procurement patterns 

4, Cost or complexity of the product 
5. Method of manufacture 

6. Predominant types of requirements 
7. Production rate 


Before deciding on a particular quality assurance system of an item, the degree 
of assurance necessary to insure that the requirements are being met must be deter- 
mined, This is the most important factor because it will directly affect decisions 
for the following: 


1. Number of requirements for which inspection is performed 
2. Precision of measurement 

3. Amount of inspection by Government personnel 

4, Extent of contractor inspection 

5. Severity of decision criteria 


The introduction of statistical techniques, offering mathematical solutions to 
industrial quality problems, particularly in the complex systems field, created the 
need for the analytical mind in the specification field. Certainly our specifications 
reflect these efforts today. 


A new problem area has arisen, however, in the administrative aspects of 
Inspection, The change in philosophy and concept which placed the inspection 
procedure in the Specification, namely Section (4), called for some reconsideration 
with regard to the approach to administratove inspection procedures. These 
responsibilities have been for the most part, spelled out in appropriate Orders and 
Handbooks, Some consideration and observations with regard to these administrative 
actions by responsible inspection organizations are offered as a result of experience 
gained since the advent of the Contractual Section (4): 
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1. A clear understanding of the intent of Section (4) should be predetermined 
prior to inspection operations. Consultation with technical personnel could resolve 
questions that could become points of conflict subsequently. 


2. It must be remembered that the Inspector is not a Design Mgineer, and 
interpretation of technical requirements must be avoided by him as well as substitute 
efforts in lieu of inadequate requirements. The Engineering Centers are responsible 
for Specifications. This includes the Design and Development of Section (4) (Quality 
Assurance Provisions). Limitations in interpretations of judgement should be 
practiced. 


3. Efficient utilization of verification procedures, which incidentally, should 
where possible, include product inspection as an initial basis for judging product 
quality. 


4, Upgrading of professionalism and maturity in Inspection Personnel is in the 
making and will no doubt be reflected in future inspection organizations, 


I mentioned earlier the utilization of Military Specifications as trouble- 
shooting documents. Two basic plans specified for military procurement in the Army 
are: 


1. Those Quality Assurance systems which are definitive in nature and are 
spelled out specifically within Section (4) of the specification. 


2. That which requires a Quality Control system to be mandatory as part of the 
procurement, 


In the first instance, MIL-I-4+5208 has been established to support the defini- 
tive type of product specification, MIJ-I-45208 establishes requirements in a 
general sense of what constitutes an acceptable inspection system. In addition it 
prescribes verification procedures that will be used by the Government to determine 
the adequacy of this system. In the second instance, MIL-Q-9858 is designed to 
support the mandatory Quality Control requirements. Certainly the trouble-shooting 
type of inspector who is looking for a place to hang his hat, so to speak, with 
regards to pinpointing trouble, can in either of the preceding specifications find 
the necessary requirement for inspection which, when not adhered to, result in poor 
product. Non-conformance to these requirements is a violation of contract; hence, a 
trouble-shooter has a very good tool for corrective action. I do not mean to imply 
that the application of MIL-I-4+5208 in itself assures that a system is adequate. 
Having an adequate system in terms of people, equipment, and space is one thing. 
Efficient operation of that system is another, Both have to be evaluated, 
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A SURVEY OF VISUAL INSPECTION 


Edward R, Close, Manager Quality Assurance 
Bausch & Lomb Incorporated, Rochester, New York 


The human visual system is, without doubt, the most extensively used inspection 
device in existence today. It is used directly as a system for testing such character- 
istics as color, shape, alignment, beauty defects, size and roughness. It is used in- 
directly in the readout system of a host of measuring systems using scales, dials, 
reticles and charts, The visual system is involved in almost all of the inspection 
decisions to accept or reject product. yet there are less than a dozen papers in 
quality control literature devoted to this subject, 


The human visual system is a most wonderful and at the same time a most ineffi- 
cient inspection device, It can detect differences in hue better than the most sensi- 
tive instruments. It can detect defects in the order of one micron in size under the 
proper conditions. On the other hand, given certain tasks, it may be subject to error 
in the order of 25% or more (1). In this day when quality costs are so important, it 
is time to bring the visual inspection system under firm control. Fry (2) has defined 
the system as having five steps: 


1. Tllumination of the object under test. 


2. Reflection of light from the object toward the eye, and transmission of light 
from the object to the eye, 


3. Admission of light into the eye and the distribution over the retina, which 
includes orienting and focusing the eye to see the object. 


4, The initiation of an impression of the object in the retina and the transmis- 
sion of this impression to the brain, 


5. The interpretation of the impression and the use of this information in per- 
forming a task. 


If we are to achieve maximum efficiency in visual inspection, we must give con- 
sideration to the entire system from the illumination of the object under test to the 
inspector's decision to accept or reject. Considering the five steps in the system in 
the order in which they occur: 


1. Diumination 


The I.E.S. Lighting Handbook (4) contains a wealth of data and information on 
this subject, and Allphin (3), Fry (2), and Weston (5) have presented further informa- 
tion which leads to the conclusion that there are at least three important factors 
which must be considered: 


a. The intensity of illumination, 
b. The wavelength composition of illumination. 
c. The contrast of the defect on the object being inspected, 


2. ed or T tt ht from the Object to th 


References (2), (3), and (4) give quite complete coverage of this step which 
leads to the conclusion that the following factors are important: 


a. The direction from which the light comes. 
b. The degree to which the light is diffused. 
ce, The lighting of the surrounding area. 

d. Peripheral sources of annoyance, 


3. Admission of light to the Eye and Focusing the Eye on the Object 


In the control of this step in the system, the opthalmologist and optometrist 
can be of greatest aid. It is well known that visual acuity changes quite rapidly 
during certain periods of an individval's life, Experience has shown that the testing 
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of the inspector's ability to perform the visual task on a testing instrument such as 
the Ortho-Rater at six month intervals is necessary to discover inspectors who may re- 
quire correction for visual acuity. 


4, Transmission of the Image to the Brain 


An adequate treatment of this part of the visual mechanism is beyond the scope 
of this discussion, 


5. Interpretation of the Image by the Brain 


It is in this step in the system where the Quality Control Engineer can make 
the greatest contribution. In order to bring this step under control it is first neces- 
sary to understand the visual inspection act which has two phases: 


a. Comparison of the product with the specification. 
b,. Making a judgment as to conformance. 


Specifications for visual defects take many forms, They may be actual samples 
of the product illustrating types of defects, or they may be prepared examples of de- 
fects such as the scratch and dig plates used as the specification for optical com- 
ponents, Klock (46) and Haynes (7). have described methods of spesifying visual char- 
acteristics in carpets and cloth. Regardless of the form which the specification may 
take, the important thing is that the decision as to what the specification is, is a 
management decision based on the goal of customer satisfaction. The decision to accept 
or reject product will be made by the inspector even if a specification does not exist. 
This decision will reflect the management policy only to the extent that he is given 
management guidance, 


The ability of an inspector to judge conformance to the specification has long 
been recognized as a major problem in the visual inspection system. Juran (8) proposed 
a plan for check inspection and mathematically assigning values to two kinds of in- 
spector error: 


a. Inspector accuracy which is the percentage of defects correctly identified. 
b. Inspector waste which is the percentage of nondefectives incorrectly iden- 
tified as defective. 


Studies have shown that, for difficult inspection tasks with good specifica- 
tions and daily check inspections, inspector accuracy is in the order of 85% to 90% 
and inspector waste is in the order of 1% to 3%, These studies were made on visual 
inspection operations which are considered to be under good control. In one inspec- 
tion operation which was not under good control inspector waste was nearly 10%, 


The problems of visual inspection are not easy to solve. They do not, however, 
constitute a dilemma, The first step in finding a solution is to recognize the visual 
inspection system in its entirety. The final solution will come through the knowledge 
of all of the sciences pertaining to the problem. These include illuminating engi- 
neering, physics, opthalmology, optometry and industrial psychology as well as quality 
control engineering. 
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DIAL INDICATOR SERVICE AND COST CONTROL 


Frank M. Dixson, Foreman, Gage Control 
Standard Pressed Steel Company, Jenkintown, Pa. 


(Ed, note) A detailed transcript of Mr. Dixson's remarks was not available 
for complete printing at the time these papers were printed. Mr. Dixson, however, 
will cover the following subjects as related to a large manufacturing concern, well 
known for its exceptionally high standards of quality combined with manufacturing 
quantities from small lot "specials" to infinitely large mass production on semi-auto- 
matic and fully automated manufacturing equipment, 


1. The Dial Indicator and its performance 
2. Checking - Range - Magnification 

3. Maintenance - Cause - Abuse 

4. Facts pertaining to Cost Control. 
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DIAL GAGE MAINTENANCE, REPAIR & CALIBRATION 


W. G. Arnold, Superintendent, Factory Engineering 
S. K, F. Industries, Philadelphia, Pa. 


In the two Philadelphia plants of S; K, F. Industries there are in daily use some 
3,500 dial indicators, The manufacturing tolerance on most of our products are finer 
than used in many tool rooms throughout the area, The metric system is used ex- 
clusively in our plants and most of our grinding tolerance are expressed in microns, 
A micron is equivalent to two and one half "tenths" (ten thousandths of an inch). To 
guarantee the high quality of precision necessary in our business it is not difficult to 
see that a first class system of Dial Indicator Gage Maintenance repair and calibra- 
tion is an absolute necessity. 


All indicators in use, or any new ones to be putin use, are numbered and a 
record established in a card file kept in the Gage Standards Room. One instrument 
maker (first class) is located in this room and spends his full time checking and re- 
pairing indicators. After the indicators are accepted from the supplier and put in use, 
periodic checks are made to see if any repairs are necessary. These check§ are re- 
corded on the record cards for the individual indicators and the frequency of the 
future checks are determined by the past history on the card, In addition, frequent 
visual inspections are required from all personnel using indicators for such obvious 
defects as broken or unstable plungers and bent or worn indicator points. 


The general periodic check schedule, unless card history dictates more fre- 
quent checks, is as follows: 


All .00lmm and .002mm indicators are inspected once a month; 
- 0001" once every two months; and 
- 005mm and .0lmm every six months 


All newly purchased indicators must pass the following "Acceptance Specifi- 
cations"'- 


(i) Repetition of Readings - Reading at any point, as for instance when checked in a 
U.D. fixture, shall be reproducible through successive movement of the spindle to 
plus or minus 1/5 graduation, 


(2) Accuracy of Calibration - The dial indicator, when checked in a master gage, 
shall be accurate to within one graduation, plus or minus, at any point of the dial from 
the approximate 10 o'clock position to the final 2 o'clock position. (Approximately 2- 
1/3 turns on Model #381 or similar type). 


(3) Sensitivity « By sensitivity we refer to the ability of a dial indicator to register as 
nearly as possible the full value of a small eccentric specimen - or a known dip in a 
Jo-Block when passed slowly under the contact point. The values should be as follows: 


For Micronar - At least 80% of a 4 micron dip = 3, 2 micron: 
For #381 or similar - At least 75% of a 6 micron dip = 4.5 micron. 


(4) Variation in Plunger Pressure - By this we refer to the difference in pressure re- 
quired to move the plunger inward as against the pressure exerted against the work 
when plunger moves outward. In all instances it always requires greater pressure to 
move the plunger inward. We are not so concerned about the actual pressure unless 
it should happen to be way out of line with other similar indicators. This test can be 
conducted in the following manner: 
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(a) Fasten indicator to bracket on special height gage stand kept in the Gage 
Room. (b) Place torsion scale under pointer and bring bracket downward by means 
of lead screw. (c) Note how many grams pressure was required before the indicator 
hand starts to move. (d) Continue the downward movement until the full range of the 
clock is reached. Notice the reading on the scale at this point (which will be consid- 
erably higher than when the hand first started to move). The difference in readings 
between these two points shows how many grams pressure was required to move the 
spindle inward its full length. (e) Reverse the travel of the bracket whereby the 
scale gradually becomes unloaded. Again notice that the scale has to be unloaded 
quite a bit before the indicator hand starts to move back, but once it starts to move it 
does not take much more unloading of the scale before the hand is back at its rest 
position. The total difference in reading at any point should not exceed 125 grams for 
#381 or similar dial indicator and 50 grams for the Micronar. 


Gage manufacturers in general do not guarantee sensitivity and plunger pres- 
sure because it is not covered in the "The American Gauge Design Specifications". 
The Standard Gage Company, however, from whom we buy most of our indicators, 
has agreed to test all indicators for S, K; F. according to the above mentioned speci- 
fications. 


In regard to Mikrokators, we do not have any specifications set up for these 
instruments because they are generally made so accurately that we have never found 
any discrepancy between any of them, However, when received they are checked on 
a special gage for calibration against a Mikrokator kept as a master in the Gage Room, 


Periodic inspection records indicate that the majority of service required falls 
in the category of cleaning, dial replacement, and crystal replacement, Our stand- 
ards ‘room, however, is fully stocked with complete spare parts inventory and our 
instrument maker is capable of making any type of repairs, It has been our experi- 
ence that, with proper care and maintenance, the life of an indicator is (for all prac- 
tical purposes) infinite. A good many of our indicators are still in use from the time 
we started keeping records about twenty years ago. 


In order to give you some idea of the cost involved in a program of this type, I 
have taken the simple expediency of dividing the sum of labor and overhead, parts pur- 
chased per year, and stationary costs divided by the number of indicators passing 
through the standards room in one year; this figure come to approximately $3. 80 per 
indicator per year. 


In view of the fact that 90% of the dial indicators used in our production and in- 
spection are of the comparator type, using accurate and expensive zeroing masters, 
the indicating mechanism is of prime importance. Therefore we consider this nomi- 
nal cost of maintenance and control money well spent, 
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COST CONTROL AND QUALITY CONTROL OF DIAL INDICATORS 


Murray P. Dwight, President 
Dwight Instrument Co., Lyndhurst, New Jersey 
(Dwight Instrument Co. is an independent repair shop specializing in dial indicators.) 


For those of you who are with companies having 1,000 employees or less, 
maintaining a quality repair shop at an economical cost becomes a problem of volume 
as well as control somewhat different from that of the large company. 


For this repair facility to prove itself economically, about six indicators per 
working day should be overhauled. This would include those which need just cleaning 
as well as those which also need parts. This number is based on an estimate of 
$100.00 per week pay for the repair man plus $150.00 per week in overhead charge, 
or a total for the year of $12,500.00. Six indicators per day then works out to around 
1,450 per year. With this volume moving through the repair shop, the cost per indi- 
cator is between $8.00 and $9.00, or close to the average charge if these indicators 
are sent to an outside repair shop for service. However, if you have less than the 
1,450 indicators needing overhaul, and if your shop is flexible and control of your work 
force is tight, you can have your repair man do jobs in the tool room or other areas 
when there is no need of indicator repair. A schedule should be maintained, and for 
each day spent, six indicators should be properly overhauled, inspected and put back 
in the gage crib; the less a man does this work, the more difficult it is to maintain a 
proper schedule of quality work. 


If, using these figures as a starting point for your cost of setting up your re- 
pair facility, you find that it is economical, you need to take the following steps; 


(1) A clean area with proper calibrating equipment must be maintained. This 
cost will be a minimum of $500. It covers the cost of benches, lights and fixtures 
and a small surface plate. Also included is a calibrating fixture which may be pur- 
chased for about $100, 00 and a set of approximately thirty gage blocks which can be 
obtained for $150. 00, 


(2) Next, a man must be properly trained. All the manufacturers of aial indi- 
cators are pleased to have your man spend a week or more in their plant, familiariz- 
ing himself with the methods and parts necessary for proper repair. 


(3) Parte needed must be ordered and an inventory system set up for these 
parts. It is from here on that your controls must be established, and these controls 
must be a part time job of men working in various departments of your plant. 


(4) The flow of indicators into and out of this facility must be the responsibil- 
ity of your gage crib. If this is not done, some indicators may enter the repair shop 
but will never come out. 


(5) Inspection must be made on each indicator after repair by the inspection 
department before it is allowed back on the shelf in the gage crib. 


(6) The inventory of parts must be kept the same way as any inventory through- 
out your plant with the repair man responsible for any undue loss. 


If you have the volume average of six indica‘ors per working day, and you are 
able to set up these controls, you can maintain a high level of quality and fast service 
on your indicators at a reasonable cost. 


Here the question arises relative to the cost as well as the inconvenience of 
sending an indicator out for repair. You will note that I have not as yet mentioned the 
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cost of ordering parts; but, if it costs money to send an indicator out for repair, it 
also costs nearly as much to process an order for parts. And in our experience, you 
often have to buy less than $10. -worth of parts, without which you cannot repair 50% of 
your indicators. Examples would be contact points, bezel screws and the like. 


If you decide to send your indicators out for repair, you have a dollar cost of 
this charge and can see exactly how much this is with respect to your over-all gaging 
program. You must, of course, maintain inspection on the indicators when they come 
back from repair, no matter where they are sent, so that you can reject then and 
there those which do not measure up. However, your rejection rate should be less 
than 1%. 


If you send these indicators back to the manufacturer, you will be assured of 
high quality work at a reasonable price. There is also the independent instrument 


repair shop which does business with many companies such as yours. 


A typical independent repair company will have five or six employees. The 
company repairs all makes and models of indicators as well as measuring instruments 
A stock of parts is maintzined to cover all the common makes. The men become 
highly skilled to do quality repair work and ¢> it fast so that the cost to you will be 
reasonable, and the repair company will still make a profit. In most respects, this 
company is operated much like the repair facility in the large plant, with the added 
cost of sales expense but a much lower overhead all told. 


There is a great variety of work which comes into these independent shops. 
One, turning out 200 items per week with a back-log of two weeks' work, will have 
150 different types of instruments to repair, One or two men, usually the foreman or 
the owner, work on the less common items which come constantly in quantities of one 
or two. This would possibly be a rarely seen foreign indicator or one of an American 
manufacture (obsolete type), twenty to thirty years old, but still in good enough con- 
dition to warrant repair. 


This independent repair company can offer you the same service as the vari- 
ous manufacturers in repair, but with some added advantages to you: 


1) All makes and models may be sent to one source, thus eliminating the pro- 
blem of splitting up the items needed for repair and sending them to the various manu- 
facturers around the country. This one source reduces to a minimum the cost of 
Paper-work and order processing within your plant. 


2) If you use a quantity of a special type indicator, the repair company will 
soon have in stock those parts needed to repair even large numbers of this particular 
type, ensnring you prompt deliveries, 


3) Cost of repair can be set up by written or verbal agreement so that you will 
know in advance how much you will be charged, on the average, for repairs, This is 
done by specifying a repair cost not to exceed, for example, 50% the price of a new 
indicator. Or if you wish to have a dollar figure fixed for each item, this can be done 
after the repair company has had some experience on the condition of your indicators 
which are sent to them, This figure could be $8.50 for your company but maybe $9, 00 
for another company. The repair company has the option to return an indicator at no 
charge if they feel that it definitely is not economical to repair. 


But whether you send your indicators back to the manufacturer or to a repair 
company, you must inspect each indicator when it is returned, and make sure that in- 
formation is properly noted on your inventory and quality control card for each indica- 
tor; obviously, it is an advantage to have a serial number on each indicator. 
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As I have discussed it here, the maintenance and repair problem (cost con- 
trol and quality control) of dial indicators may be summed up as follows: 


(1) If your plant has a volume of around 1,450 indicators per year which need 
cleaning or repair, a shop in your own plant may be well justified economically. You 
must, to maintzin quality, have an experienced man whose work is inspected by an- 
other person and the flow of repairs should be controlled by another person to insure 
that an indicator will not get into the repair room and never come out, 


If these conditions cannot be met it is expedient to send all repairs to an out- 
side shop, either to the manufacturer or to an instrument repair shop. When this is 
done, all of the above requirements are carried out automatically; (1) you have the 
paper work on what indicators are out and how long they have been out; (2) you do not 
have to maintain a stock of parts with the corresponding cost of inventory records 
needed to control this stock; (3) the indicators are inspected when returned before 
going into stock and (4) you then have, on paper, a control of cost and quality on your 
dial indicator servicing. 
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MEN--CIVILIZATION--MEASUREMENT 


M, C, Curtis, General Foreman, Tool Inspection 
The Timken Roller Bearing Company, Canton 6, Ohio 


Measuring or gaging is a very important part of manufacturing. This 
discussion will be limited to linear and angular measurements. 


History indicates that as we improve our ability to seasure, our 
standard of living improves. 


In the 14th century, parts of the human body were used as standards 
of length. The first 16 men to leave church on Sunday morning were 
lined with the left foot, heel to toe, and the distance from the toe of 
the first person to the heel of the last was considered the legal rod. 


The cubit was one of the first known measurements. We are told that 
there were two -- one 18" and one 20". Two-thirds of the 18" cubit was 
divided into 12 equal parts and called an inch. Obviously standards of 
this type would not suffice for present day practice. 


All measurements are relative. They must be compared with some 
known standard; for example, a steel rule, a precision gage block, or 
a light wave. 


Unless we can describe a manufactured part in terms of numbers, it 
cannot be fully described. It is not enough to say that it is large or 


FIGURE 1 -- The Timken Company Gage Laboratory is the best equipped 
industrial gage laboratory in the United States. This laboratory is 
temperature, dust and humidity controlled. Thirty people are employed 
here. 
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FIGURE 2 -- Individually wired desks are used for work benches in the 
Timken Company Gage Lab. One hundred and twelve people are employed in 
the entire Tool Inspection Department which controls approximately 
500,000 tools and gages including 4,000 reference gages. 


small or round or square; it must be identified in terms of inches, 
angles, radii, and so forth. 


All things man made must be measured. They must be measured in 
some manner, although some measurements are crude and some are refined. 
Highly precise measurements today may be considered crude in the future. 


George Washington University, one of our leading institutions of 
higher learning, has formally recognized the importance of accurate 
measurements by offering a course on the subject. I think it is 
timely and will make possible, for those who are interested in measuring, 
an effective means of obtaining information on the subject that would 
be difficult to obtain in the usual manner. 


The difficulty of training men in the skill of measuring can be 
appreciated when I tell you that we obtain our tool inspectors through 
an apprentice training program requiring four years. This training 
program takes the apprentices through all departments of our bearing 
factory, including the tool cribs, screw machine departments, grinding 
department, inspection department, tool room and the engineering 
department. They finish in the gage laboratory. It is here that the 
patient, painstaking task of measuring parts accurately begins. 


Parts must be measured to the proper degree of accuracy. We would 
not attempt to measure a part 100" diameter to a tolerance of 10 
millionths. We would not measure a rough casting with a highly precise 
gage that might be graduated in terms of millionths. It would not be 
good practice to attempt to measure a highly precise part with an 
instrument that would read in terms of fractions of an inch; therefore, 
it becomes necessary to select the proper equipment for the job. 
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We use for a tolerance on our 
masters and gages an unwritten rule 
of 1/10 of the permissible toler- 
ance allowed on the part to be 
gaged. If the part has a permis- 
sible tolerance of 1 ten-thousandth 
of an inch, the tolerance on the 
master or setting gage should be 10 
millionths of an inch. By compar- 
ison, we have made many gages over 
the past several years to this 
degree of accuracy. We are 
fortunate in dealing mostly with 
tapered parts that make it possible 
for us to work to this degree of 
accuracy, that is, 10 millionths of 
an inch. Usually we can grind a 
gage round and straight, and with 


FIGURE 4 -- A Bausch & Lomb uni- 
versal projector is used in checking 
a wide variety of jobs including 
threads, radii and profile gages. 


PENNSYLVANIA 


FIGURE 3 -- The Zeiss Universal 
Measuring Microscope is considered 
most versatile gage in the Timken 
Company's Gage Laboratory. The 
machine is used primarily for 
checking accuracy of tool profile 
gages necessary for grinding of 
orming tools used on automatic 
screw machines. 


the proper surface finish on a 
tapered part, then face until we 
arrive at the proper size. 


Go and No-go gages have their 
place in the business and they are 
economical to manufacture, but may 
be costly, too, in the long run. 

A Go or No-go plain plug gage will 
either enter a hole or it will not 
enter. If it does enter, it will 
not indicate out of round that may 
be present. It is difficult to 
determine the amount, if any, of 
bellmouth or taper and will not 
indicate the difference that may 
exist from basic size. Indicating 
type equipment could show the 
amount of out of round, taper, or 
bellmouth and show the difference 
between the size and basic size. 
Indicating equipment makes it 
possible to set cutting tools to 
the best advantage. As the tool 
wears and causes the part to in- 
crease or decrease in size, it can 
be set to obtain a maximum tool 
life before sharpening. 


Progress in our ability to 
compare parts with master gages or 
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precision gage blocks has improved considerably in the last few years, 


Many types of comparator gages such as dial indicators, optical 
comparator type gages, pneumatic gages and electronic gages have been 
developed to a _ degree of accuracy. The mechanical indicator is 
used extensively for many operations. It is a very reliable, dependable, 
economical, and practical method of comparing and gaging manufactured 
parts. 


Pneumatic type gages have their place in comparing parts with a 
part of known size, shape, and accuracy. They are especially useful 
where it is possible to fixture for the gauging operation. They are 
well suited for comparing parts with thin walls, parts with a highly 
polished surface. The life of the master or setting gage is extremely 
good because of the fact that it will not become scratched and is 
subject to little wear. Pneumatic gages are especially adapted to 
comparing parts with a tolerance of 5 thousandths of an inch or less. 
Where tolerances are less than 1 ten-thousandth of an inch, electronic 
gages are more practical. 


There are many electronic gages built today with a high degree of 
accuracy and reliability. Electronic gages are best suited where 
tolerances are in the order of the low millionths of an inch. We use 
many electronic gages throughout our plant on the production line as 
well as in our gage laboratory. Electronic gages are not 10 
electronic. Since pickup is usually mechanical, the gage will be only 


FIGURE 5 -- The Talyrond is a gage designed for checking roundness of 
machine parts to an accuracy of one millionth of an inch, It not only 
shows the degree of roundness but it also indicates the shape of the 
out-of-round part. 
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FIGURE 6 -- On this Graph-Mo surface plate, the operator is checking a 
roller master for size and taper. At left (rear), Graph-Mo ring gage 

is checked for length. Cone taper gage is checked for taper in center 
foreground. It is one of the flattest plates of its size in the world -- 
within 50 millionths. 


as good as the mechanical part of the gage. 


We have cylindrical grinders for grinding both inside and outside 
diameters. They are capable of grinding to round within 2 millionths 
of an inch, straight to the same degree of accuracy, with surface 
finish as low as 1 micro inch, 


Four factors are required to achieve this high degree of grinding 
accuracy: We must have a machine with this capability; have an effi- 
cient operator; the environment must be right, and we must have the 
proper material with which to work. We could not expect to attain this 
degree of accuracy with a machine that is not in good condition. The 
machine must be as nearly free from vibration as possible, table ways 
must be straight, motor and motor pulleys must be properly balanced, 
the wheel must be dressed where it contacts the work, a good diamond 
must be used. The dressing diamond should be a ground conical diamond. 
The operator should be sufficiently skilled to obtain from the machine 
its full capability. The environment must be right, or we cannot expect | 
to grind to a high degree of accuracy. 


The machine must be mounted on a vibration-free base; the 
temperature should be controlled in a room with a minimum amount of 
noise, The room must be as free from vibration as possible, as 
vibration is one of the greatest enemies of high precision grinding. 


Steel is not an inert material. It changes size and shape with a 
change of temperature. This is characteristic of steel and cannot be 
altered, It is resilient, it sometimes sags from its own weight. A 
cylindrical piece will deform elastically when it lays on its 
periphery, therefore, care must be exercised in order to avoid dis- 
tortion. Distortion can also result from too much gauging pressure or 
by the way in which the piece is held during the gauging operation. 
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FIGURE 7 -- A SCIP MUL 214 with a 4 x 16-inch capacity, universal three 
dimensional measuring machine. It will measure the pitch diameter, 
round, straight and bellmouth of internal threads. Accurate to within 
twenty millionths of an inch. 


Temperature is very important. Unless the piece is gaged at the 
international temperature of 68°, or unless the temperature is known 
and compensated for, we cannot expect to obtain the proper degree of 
accuracy. Steel changes at the rate of 6-1/2 millionths per inch per 
degree. 


Stress becomes a very important factor in obtaining a high degree 
of accuracy. Materials should be as nearly free from stress as 
possible. We are aware of the fact that we set up stress in the piece 
when we grind. At the same time, we may relieve stress during the 
grinding operation. It is possible to finish one part of the piece 
to the required degree of accuracy only to have it change when another 
part of the same piece on another operation may cause the first part 
to change. We used to believe that we were relieving stress during a 
hand lapping operation. We have found out now that we do not relieve 
stress but produce more stress during the hand lapping operation. 
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Foreign particles such as dust, dirt and lint, always present, are 
one of our greatest enemies in trying to achieve high precision gauging. 
Every effort should be made to have the part to be gaged as nearly 
clean as possible. 


Surface finish is another important factor in gauging to a high 
degree of accuracy. We cannot hope to measure in the low millionths 
a surface whose finish is rough and irregular. In order to determine 
the size, it is necessary for it to be round, straight, and free from 
irregularities. This is true of cylindrical work. The same applies 
to flat work. Unless it is flat and the surface sufficiently 
regular, free from taper, we cannot expect to measure to a high degree 
of accuracy. 


Surface finish is probably the most difficult measurement. Most 
surface finish checking gages do not show waviness. It is difficult 
to determine where roughness leaves off and waviness begins. Few if 
any of our surface measuring instruments show the exact profile of 
the surface finish. Surface finish can differ in profile and it will 
not be shown on the read-out. Surface finish measurement is not a 
true measurement. It is an average deviation in height as determined 
by a stylus with a 5 ten-thousandth radius and propelled over the 
surface at a speed that will determine the roughness width cut off. 
The reading might change radically with a change in roughness width 
cut off. The vertical magnification is much higher than the 
horizontal magnification, Since the read-out is distorted, it is not 
a true indication of the surface finish profile. 


In order to obtain the desired accuracy on a finished part, it 
is necessary to start as nearly precise as possible. It is necessary 
to be as nearly right on the absolute check as possible. Each time a 
comparison is made, there is a possibility of losing accuracy. (If 
we err on the absolute check with an interferometer when checking 
precision gage blocks, then err between the gage block and the master 
gage and again err between the master gage and setting the comparator 
and again when the gage is read, then we may finish with all these 
errors in the final result.) 


I have mentioned the fact that we have improved our ability to 
machine, and that we have comparators capable of a very high degree of 
accuracy. Until 1952, however, little progress had been made in 
precision gage block accuracy. The tolerances at that time for 
precision gage blocks had not improved in over 25 years. The tolerance 


FIGURE 8 -- The Link 
fringecount micrometer: a 
combination interferometer 
and electric counter. It 
is possible to determine 
absolute length to an 
accuracy of one millionth 
of an inch and can measure 
lengths up to two inches. 
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for "AA" quality blocks was plus or minus 2 millionths; for "A" 
quality, plus or minus 4 millionths, and for "B" quality, plus or minus 
8 millionths of an inch. We felt that "AA" quality precision gage 
blocks (the practical standard used through the worid), should be 
within 1/2 millionth of an inch. In order to accomplish 1/2 millionth 
accuracy in precision gage blocks, the absolute interferometer should 
have capabilities in the order of 1 ten-millionth of an inch to 
accomplish 1/2 millionth measurements. The project to improve 
standards of accuracy was started at that time by a group of interested 
manufacturers. Money was received from the congress and the job was 
undertaken by the National Bureau of Standards in Washington. The 
project is now under way and will take a few years to complete. In 

the meantime, manufacturers have produced precision gage blocks to a 
guaranteed accuracy of 1 millionth of an inch. 


For the past 70 years our standard for length has been a platinum- 
iridium bar on which two lines are scribed one meter apart. The number 
of light waves that appear between these two lines were counted and 
a value assigned to the light wave. There were a number of light 
sources that could be used for this purpose. Recently it was agreed 
internationally that the orange-red line in krypton 86 will be used 
in the future, ae this light source, there are 1,650,763.73 light 
waves in one meter of length. 


The interferometer is an optical gage with a monochromatic light 
source with which we can determine the absolute length. Our Bureau 
of Standards has partly completed an interferometer that it is hoped 
will have a capability of measuring to 1 or 2 tenths of a millionth. 


Separating the concept of a rod as a unit of measure and the 
interferometer is six hundred years. Over that period of six centuries 
a constant refining process has taken place. Within the past twenty 
years important steps have been taken to further refine our ability 
to measure accurately. Positioned as we are on the threshold of space 
exploration and travel, accuracy takes on even greater importance. It 
may well be that the ability to measure accurately can spell the 
difference in the mighty struggle between East and West that is taking 
place, even as we meet here. Thank you. 


ADVANCED TRENDS IN ELECTRIC GAGING 


Robert A. Dana, Manager, Industrial Products Sales 
Hamilton Watch Company, Lancaster, Pennsylvania 


Two years ago, Industrial Products personnel at the Hamilton Watch Company reali- 
zed there was a growing demand for a simple, highly reliable, and very accurate Gaging 
System that could be applied to industrial processes to reduce the cost of parts fab- 
rication and inspection. Developmental work was undertaken, at that time, and the re- 
sult today is a system known as Electric Gaging. This paper will define Electric Gag- 
ing; it will describe the function of the Electric Indicator and control units; and 
it will describe typical applications of Electric Gaging to reduce the cost of parts 
fabrication and inspection. 


DEFINITION OF ELECTRIC GAGING 


Electric Gaging is a Measurement System, utilizing an Electric Indicator and suit- 
able control equipment to provide an electrical signal whenever pre-established dimen- 
sional limits or tolerances are exceeded. The Electric Indicator portion of the system 
is a high precision limit switch which, when adequately fixtured, is capable of measur- 
ing a particular dimension on a part in providing a switch closure when the actual dim- 
ension measured is above or below pre-established limits. To convert the indicator 
switch closure to a useful output signal, a control unit is used in conjunction with 
the Electric Indicator. Selection of the control unit depends upon the requirements of 
the application in question, 


FUNCTION OF THE ELECTRIC INDICATOR 


The Electric Indicator resembles a 
Mechanical Dial Indicator in that it has a 
spindle, gear train, and rather conventional 
housing. It differs from a Mechanical In- 
dicator in that two adjustable electrical 
contacts are built into the movement to pro- 
vide the switching actions. Also, there are 
two adjusting screws (high and low toleran- 
ce) to set the dimension at which switching 
action will take place. 


Technically, the Electric Indicator is 
a high precision SPDT switch. It differs 
from an ordinary switch in that there is an- 
plification (approximately 100X) between the 
activating mechanism (spindle in this case) 
and the switch contacts. This amplification 
is the reason the Electric Indicator will re- 
peat a dimensional reading electrically to 
better than ten-millionths of an inch, 


An ordinary set of switch contacts will 
repeat a dimensional reading electrically to 
better than 0,001", therefore, it is safe to 
assume that the contacts in the Electric In- 
dicator repeat at least that well. Between 
the contacts and the spindle there is 100X 
amplification through the gear train and, 
therefore, the Electric Indicator will re- 
peat to ten-millionths of an inch (0,001" ¢ 
100 = 0.00001"), 


Laboratory and field tests of the Elec- 
tric Indicator have shown that the unit will 
repeat electrically to better than ten-mill- 
ionths over extended periods of use, FIGURE 1 -- Electric Indicator 
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FUNCTION OF THE BASIC LIGHT BOX 


The Basic ight Box is designed to be 
used in conjunction with the Electric In- 
dicator to convert a switch closure in the 
Electric Indicator to a visual signal (con- 
sisting of three lights) that can be seen 
by an operator. If the measurement being 
taken is either higher or lower than the 
pre-established limits, the appropriate 
"out-of-tolerance” light will go on. 


To turn on any one of three lights in 
the Basic Light Box with the SPDT switch in 
the Electric Indicator is somewhat of a 
trick without using auxiliary switching de- 
vices, such as relays, This problem is sol- 
ved with the use of a low voltage bridge 
circuit. Figure 2 is a Schematic Diagram of 
this circuit. A switch closure in the Elec- 
tric Indicator will add a resistance (out- 
of-tolerance lamp) in parallel with the fix- 
ed resistor and the "in-tolerance" lamp. 
With this added resistance, the voltage drop 
across the “in-tolerance" lamp is reduced 
nearly to zero and the light goes out. 


FUNCTION OF THE AMPLIFIER CONTROL BOX 


In automatic gaging and machine control 
applications, where it is necessary to 
switch voltage and current in excess of the 
safe values for the contacts inside the 
Electric Indicator, an amplifier is employed 
so that a very low level signal from the 
Electric Indicator will activate a relay 
capable of switching 10 amps at 110 volts, 
resistive, 


Figure 3 shows a dual channel vacuum 
type amplifier commonly employed in avt>- 
matic gaging. A similar amplifier, vu‘ <lizs- 
ing two transistors instead of the tube, is 
available for customers who prefer this 
type. The application of the two units is 
identical, 


There are several other circuits 
available for control applications where 
memory, counting, or timing functions are 
required, They will not be described be- 
cause their use depend upon specific gaging 
or control problems. 


APPLICATIONS OF ELECTRIC GAGING SYSTEMS 
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FIGURE 2 -- Basic Light Box 


Electric Gaging can be applied in two basic ways. 
then measure its dimensions, either manually or automatically, The other is to integra- 
te Electric Gaging with a machine tool to control size as the part is being fabricated. 
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ELECTRIC INDICATOR 


FIGURE 3 -- Amplifier Control Box 


One is to fabricate a part and 


In the first method, "in-tolerance” parts are separated from “out-of-tolerance" 
parts while in the second method only "in-tolerance" parts can be fabricaied (assuming 
the machine tool is functioning properly). It is fairly certain that there is a demand 
for both types of Electric Gaging, therefore, the applications will cover both major 
areas, 
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POST PROCESS GAGING 


ELECTRIC 
INDICATOR 


BASIC LIGHT BOX 


FIGURE 4 Manual Gaging 


Electric Gaging can be used to advantage in measuring either single or multiple 
dimensions on a part after it has been fabricated. Figure 4 shows a typical applica- 
tion where a part is hand fed into a measurement station and the operator reads lights 
(over, good, under) instead of a dial. There are several advantages to using Electric 
Gaging on this type application when compared with using a conventional dial gage. Not 
only can an operator read lights faster than he can read a dial, but nearly all judge- 
ment is removed from the job because the Gaging System decides whether the dimension 
being measured is "in" or "out-of-tolerance”. 


MACHINE CONTROL 


ELECTRIC INDICATOR 


SORTING 
STATION — 
, OVER SIZE 

Z- ON SIZE 

— UNDER SIZE 


FIGURE 5 -- Automatic Gaging & Sorting 


Figure 5 shows another application of post-process gaging where the Electric Gag- 
ing System is used in conjunction with automatic feeding and sorting devices. For ex- 
ample, parts can be hopper fed into a gaging station where one or more dimensions are 
measured, and then the part automatically sorted into two or three categories. 
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As stated above, Electric Gaging can be applied directly to machine tools to con- 
trol the size of a part as it is being machined, Figure 6 shows one application where 
Electric Gaging is used to measure stock dimensions before it enters a machining pro- 
cess, For example, the thickness of strip material is measured before entering a punch 
press operation. If the thickness of the material is out-of-tolerance the Electric 
Gaging System will shut off the machire, thereby, preventing damage to the tools or 
preventing out-of-tolerance parts, 


ELECTRIC 
-— MACHINE 
CONTROL 
UNIT 


/ TO MACHINE 
CONTROL 


— GRINDING 
WHEEL 


FIGURE 7 -- Size Control on a Machine Tool 


A second application of Electric Gaging to machine control is show in Figure 7, 
In this example, the Electric Gaging System is constantly measuring the part as it is 
being machined and will activate the next machine cycle when the part is on size. 


CONCLUSION 
From the above discussion of Electric Gaging, it can be seen that there is a 


place in industry for this Gaging System wherever dimensions must be accurately con- 
trolled. This Gaging System can be economically justified in manual gaging operations, 


automatic gaging, and automatic size control on a machine tool, It is hoped that this 
discussion of Electric Gaging will help Manufacturing Management personnel become more 


cognizant of the fact that this system is one effective way of reducing manufacturing 
costs. 
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